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1.  Platform  Level  (Option  Phase)  Introduction 

This  report  summarizes  recent  progress  made  on  the  AVIATR  contract  Task  Order  3, 
Condition  Based  Maintenance  plus  Structural  Integrity  (CBM+SI)  -  Option  Phase.  Similar  to 
previous  progress  reports  during  the  Option  Phase,  the  report  in  general  follows  the  process 
detailed  in  the  flowchart.  A  detailed  general  purpose  flow  chart  is  presented  in  Section  3. 
This  is  followed  by  a  detailed  application  for  the  general  process  to  our  specific  application 
on  the  F-15  wing  demonstration  in  Section  4.  The  remaining  sections  tackle  specific  topics 
within  the  flowchart  that  have  been  tasked  during  the  reporting  period.  In  this  progress  report 
tasks  related  to  in-situ  sensor  system  capability  analysis  (Section  5),  updates  to  the 
component  level  risk  analysis  (Section  6)  and  cost  benefit  analysis  (Section  7)  are  discussed. 

The  discussion  in  Section  5,  ”In-situ  Sensor  Capability  Analysis”  is  a  summary  of  the 
progress  made  to  understand  and  represent  the  detection  capability  of  Structural  Health 
Monitoring  (SHM)  technologies  that  could  be  employed  on  the  F-15  application  under 
consideration.  As  a  starting  point,  it  is  based  on  the  methods  and  assumptions  defined  in  the 
Department  of  Defense  Handbook  on  “Nondestructive  Evaluation  System  Reliability 
Assessment”  (MIL-HDBK- 1823 A). 

Section  6,  discusses  the  modifications  that  have  been  made  to  the  general  risk  analysis 
implementation  in  the  RBDMS  code  and  the  detailed  analysis  for  the  subset  of  16  locations 
considered  initially..  Similarly,  Section  7  continues  the  development  of  the  Cost/Benefit 
Analysis  (CBA)  that  was  started  in  Phase  I  with  the  expansion  to  consider  a  number  of 
additional  requirements  i.e.,  field  inspections  vs.  Depot  inspections,  false  call  rates,  etc.  As 
well  as  expanding  the  scale  to  include  a  number  of  different  locations  and  application  to  the 
16  location  subset. 
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2.  Final  Control  Point  Selections 


2.1.  March  Status 

The  March  2011  progress  report  closed  with  four  (out  of  a  possible  twelve)  aircraft 
sections  suitable  for  the  system-level  demonstration:  1)  forward  center  fuselage,  2)  aft  center 
fuselage,  3)  inner  wing  and  4)  outer  wing.  These  primary  structural  candidates  contained  the 
necessary  information  to  warrant  credible  and  comprehensive  analyses  as  defined  by  this 
project’s  goals.  From  this  list,  the  chosen  candidate’s  control  points  would  then  be  identified 
and  the  corresponding  damage  tolerance  data  applied  to  the  risk  and  cost  benefit  analyses. 

2.2.  Progress  and  Resolution 

Final  selection  of  the  remaining  candidates  depended  on  several  factors  including  the 
number  of  critical  locations,  access  variability  of  control  points,  ample  history  of  in-service 
flaws,  and  existing  F-15  Program  risk  analyses  which  pinpointed  numerous  high  risk 
locations.  Further  research  revealed  that  both  the  inner  and  outer  wing  together  formed  an 
excellent  example  system. 

2.2.1.  Consolidation 

Table  1  lists  the  control  point  tally  of  the  Wing  segments  along  with  the  associated 
“criticality”.  As  defined  in  the  March  progress  report,  the  criticality  of  an  item  is  based  upon 
the  severity  of  its  impact  to  the  aircraft  should  the  item  fail.  Direct  and  Indirect  items  (blue 
and  purple  coded  parts)  refer  to  items  that  have  a  significant  effect  on  the  airworthiness  of 
the  aircraft.  Durability  Critical  items  (orange  coded_  parts )  refer  have  a  relatively  minor 
impact  on  the  aircraft  and  immediate  replacement  (though  needed)  is  not  paramount. 
Additionally,  items  not  established  as  critical  in  the  FSMP  report  were  re-addressed  in  the  F- 
15’s  Program’s  Silver  Bullet  Risk  Analysis  report  (Document  LF08-084)  which  attempted  to 
assess  the  risk  of  failure  for  the  existing  damage  tolerance  parts.  Data  within  this  document 
provided  additional  information  of  higher  risk  parts  that  were  not  previously  considered 
critical  in  the  FSMP  report.  Incorporating  both  the  C/D  and  E  models,  the  control  point  grand 
total  within  each  section  is  summed  below. 
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Candidates 


Inner  Wing 


Aft  Center 
Fuselage 


Outer  Wing 


Forward 

Center 


CONTROL  POINT  TALLY  BEFORE  A/C  MODEL  DISECTION 

(C/D  &  E) 

FSMP  "Items  of  Criticality"  Lists 
(Serv  Life  <  32K  FH) 

FST 
(DC) 


Risk 

Document 

Grand 

Total 

Sub 

Total 

<32  K 
(FH) 

>  32K 
(FH) 

54 

3 

9 

66 

24 

47 

0 

2 

49 

12 

Table  1.  Control  Point  Tally  for  System-Level  Demonstration 


2.2.2.  Manipulation 

Additional  steps  were  performed  before  the  final  control  point  number  was  determined. 

The  in-service  (INS)  and  full-scale  test  (FST)  items  (brown  and  green  colored  cells) 
were  ignored.  Neither  classification  possessed  the  necessary  information  to  achieve  a 
comprehensive  risk  analysis.  Thus,  both  were  discarded. 

The  control  points  of  each  wing  section  were  divided  into  C/D  and  E  models.  In  doing 
so,  a  sub-sectional  risk  vs.  cost/benefit  analysis  could  be  performed  based  on  aircraft  version, 
establishing  a  more  realistic  comparison.  As  of  this  writing,  only  the  C/D  control  points  have 
been  applied  to  this  research. 

From  these  steps,  the  left-sided  portions  (Grayed  cells)  of  Table  2  and  Table  3  itemize 
the  new  arrangement.  The  column  “Total  Before  Triage”  lists  the  control  point  tally: 

C/D  model  Wing  Sections  E  model  Wing  Sections 

Inner:  35  control  points  Inner:  27  control  points 

Outer:  31  control  points  Outer:  18  control  point 

One  note:  Some  DTA  items  documented  in  the  FSMP  report  possess  distinctions  within 
the  item  itself.  Several  E  version  DTA  items  separate  damage  tolerance  information  between 
aircraft  tails  (ex:  Tails  1-209  versus  Tails  210+;  both  represented  by  the  same  DTA  number). 
At  this  point,  there  is  no  differentiation.  A  reassessment  of  this  issue  will  occur  should  the  E 
model  become  part  of  the  system-level  risk  analysis. 
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CONTROL  POINT  DOWN-SELECT  EVOLUTION 
(C/D  Model) 


Candidates 

FSMP  "Items  of  Criticality"  Lists 
(Serv  Life  <  32000  FH) 

Risk 

Document 

Total 

Before 

- ' 

Triage 

- * 

Projet 

Control 

DIR 

IND(L) 

i  i 

;  ind 

I  I 

i  DC 

INS  !  FST  '  FST 
®  !  (SOF)  i  (DC) 

Sub 

Total 

<32  K 
(FH) 

>  32K 
(FH) 

Triage 

Non 

DTAs 

Adcf  1 

Elim 

Points 

Inner  Wing 

8 

2 

2 

14 

n/a  n/a  n/a 

26 

3 

6 

35 

1 

5 

29 

Outer  Wing 

4 

1 

7 

18 

n/a  n/a  n/a 

30 

0 

1 

* 

31 

8 

1 

22 

SUM  -> 

66 

9 

6 

51 

Table  2.  Control  Point  Count  for  Wing  Segments  (C/D  model) 


CONTROL  POINT  DOWN-SELECT  EVOLUTION 


Candidates 

FSMP  "Items  of  Criticality"  Lists 
(Serv  Life  <  32K  FH) 

Risk 

Document 

Total 

before 

Triage 

Triage 
(none  yet 
performed) 

Projet 

Control 

Points 

DIR 

IND(L) 

1  1 

:  ind  : 

i _ i 

1  I 

:  dc  : 

i _ i 

!  FST  |  FST 
_ S  i  (SOF)  ;  (DC) 

Sub 

Total 

<32  K 
(FH) 

>  32K 
(FH) 

Non 

DTAs 

Adcfl 

Elim 

Inner  Wing 

8 

6 

10 

0 

n/a  n/a  n/a 

24 

0 

3 

27 

0 

0 

27 

Outer  Wing 

8 

2 

2 

5 

n/a  n/a  n/a 

17 

0 

1 

18 

0 

0 

18 

*  SUM  ->| 

45 

0 

0 

45 

Table  3.  Control  Point  Count  for  Wing  Segments  (E  model) 


2.2.3.  Triage 

Once  the  down-selection  process  was  complete,  each  control  point  underwent  a 
“triaged”  health  examination.  This  step  served  two  distinct  purposes:  1)  determined  whether 
or  not  the  control  point  met  a  specific  minimum  requirement  defined  for  this  project  and  2) 
established  and  prioritized  the  risk  sensitivity  of  those  remaining. 

The  requirement  key  to  this  juncture  was  that  the  Single  Flight  Probability  of  Failure 
(SFPOF)  for  a  control  point  never  exceeds  1 -in-ten-million  at  any  given  time  during  aircraft 
operation.  If,  at  any  flight  hour,  the  control  point  exceeds  this  threshold,  then  the  risk  to  the 
aircraft  was  deemed  too  high  for  operational  use. 

To  facilitate  “triage”,  damage  tolerance  data  derived  from  the  FSMP  report  (and  from 
the  F-15  Program  community)  served  as  inputs  into  the  RBDMS  code  to  map  the  inspection 
timelines.  The  location  of  this  timeline  (specifically  the  slope  of  the  curve  in  relation  to  x- 
and  y-axis  thresholds)  helped  determine  the  sensitivity  of  the  control  point  to  risk  analysis. 
(Details  of  this  relationship  are  discussed  on  Section  6.1:  Overview  of  RBDMS.)  The  control 
point  was  discarded  from  the  study  if  one  of  two  conditions  occurred: 

•  If,  at  any  time,  the  SFPOF  for  the  control  point  was  greater  than  10~7  (1 -in- ten- 
million)  AND  the  first  inspection  interval  exceeded  the  maximum  service  life  of 
the  aircraft  (18000  flight  hours),  then  the  control  point  was  deemed  insensitive  for 
the  purposes  of  this  CBM+SI  study. 
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•  If  the  first  inspection  interval  never  exceeded  the  10'7  SFPOF  threshold  beyond 
the  aircraft’s  service  life  (18000  flight  hours),  then  the  item  was  deemed 
insensitive  as  well. 

The  column  “Triage”  listed  in  Table  2  and  Table  3  specify  the  number  of  control  points 
in  the  Wing  segments  which  were  excluded  from  the  risk  analysis.  Because  the  E  model  is 
not  yet  considered  in  this  study,  triage  was  not  performed.  However,  the  C/D  model  has 
undergone  a  complete  triage  with  a  total  of  15  items  removed  from  contention.  Non-DTA 
items  are  those  control  points  which  the  F-15  Program  deemed  inconsequential  to  the 
airworthiness  of  the  aircraft  should  these  areas  fail  (if  they  indeed  ever  do).  Of  these 
“uncounted  9”,  one  resides  in  the  inner  wing  while  8  reside  in  the  outer  wing.  The  column 
labeled  “Add’l  Elim”  (or  Additional  Eliminations)  represents  the  control  points  involved  in 
triage  and  were  eliminated  based  on  one  of  the  two  conditions  listed  directly  above.  Five  (5) 
control  points  were  removed  from  the  inner  wing  risk  analysis  and  one  (1)  removed  from  the 
outer  wing.  The  remaining  control  points  all  were  deemed  sensitive  enough  for  continuation 
within  this  study. 
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3.  CBM+SI  Process  Flowchart 

3.1.  Purpose/Introduction 

Slated  as  a  deliverable  is  a  flowchart  depicting  the  condition-based  maintenance 
process  which  the  structural  integrity  analysis  follows.  The  flowchart  focuses  on  the 
application  of  probabilistic  risk  analysis  to  an  existing  aircraft  design,  along  with  the  optional 
application  of  a  specific  risk-mitigation  technology  (i.e.  in-situ  sensors).  The  purpose  is  to 
provide  a  blueprint  that  any  organization  can  duplicate  to  arrive  at  a  feasible  and  cost- 
effective  solution. 

The  flowchart  presented  in  this  section  is  intended  to  be  generally  applicable  in  a 
CBM+SI  setting.  In  this  project,  the  example  failure  mode  being  considered  is  the  growth  of 
fatigue  cracks  in  metallic  structure  resulting  in  unstable  crack  growth.  However,  the  process 
being  developed  here  could  be  used  for  other  failure  modes,  such  as  composite  disbond, 
corrosion,  etc.  In  addition,  this  project  investigates  the  use  of  in-situ  fatigue  crack  detection 
sensors  for  mitigation  of  risk.  Any  technology  which  is  capable  of  reducing  the  risk 
associated  with  the  failure  mode  under  consideration  could  be  applied  here,  such  as  disbond 
sensors,  a  new  Non-Destructive  Evaluation  (NDE)  technique,  an  unconventional  repair 
method,  etc. 

Note  that  because  of  the  evolving  work  performed  on  this  contract,  the  flowchart  is 
under  constant  revision.  As  such,  the  paradigm  will  be  defined  at  the  writing  of  each  progress 
report.  Hence,  the  August  2011  update  defines  the  flowchart  as  it  stands  at  this  time.  Also,  it 
is  the  intention  of  the  team  to  produce  detail  flowcharts  that  are  specific  to  the  in-situ  sensor 
capability  analysis,  risk  analysis,  and  cost-benefit  analysis  sections  of  this  report.  However, 
these  have  yet  to  be  produced. 

In  the  following  section  the  process  flowchart  shown  here  will  be  applied  to  our 
specific  problem  from  start  to  finish  using  a  sixteen  location  subset  of  the  total  list  of  control 
points  from  the  wing  system.  This  will  facilitate  the  communication  of  the  use  of  the 
flowchart,  as  well  as  represent  the  main  narrative  of  the  analysis  as  it  stands.  In  later 
sections,  each  major  component  of  the  analysis  (in-situ  sensor  capability  analysis,  risk 
analysis,  cost  benefit  analysis)  is  extensively  detailed. 

3.2.  Flowchart 

The  following  flowchart  describes  the  generalized  strategy  to  perform  optimal 
condition-based  maintenance  (CBM)  analysis  with  an  emphasis  on  risk  analysis  and  an 
optional  risk-mitigation  technology.  This  includes  the  definition  of  the  problem, 
establishment  of  user-defined  parameters,  research  and  gathering  of  data,  as  well  as  the 
various  analyses  that  must  be  performed  to  assist  in  the  decision  making  process.  Figure  1 
displays  the  model. 
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Figure  1.  Condition-Based  Maintenance  plus  Structural  Integrity  (CBM+SI)  Process 

Flowchart 


3.3.  Flowchart  Components 


Each  flowchart  step  in  Figure  1  is  explained  below. 


4 


\ 


Establish  Analysis 
Framework 


Step  1)  The  strategy  for  controlling  a  system’s  health  must 
abide  by  the  definitions,  Technical  Performance  Measures 
(TPM),  and  analyses  parameters  that  the  user  has  established. 
Without  this  bounding,  determining  an  optimal  strategy  is 
ineffectual.  The  framework  developed  is  failure  mode-specific 
and,  as  such,  must  remain  constant  throughout  the  entire  analysis 
process. 
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Acquire  Structural 
System  Information 
of  Article 


Step  2)  Acquiring  structural  system  information  refers  to 
any  analysis  information  common  to  a  platform’s  fleet.  Data 
include  fleet  size  and  service  life,  average  hours  per  flight, 
control  point  locations,  load  spectrums  and  any  other  criteria 
which  are  independent  of  tail  number  or  unit. 


Step  3)  Data  is  collected  from  all  control  points  relevant  to 
Acquire  Control  the  analyses.  Since  this  step  is  part-specific,  drawings  and/or 

Point  information  parts  lists  may  be  obtained  for  material  characteristics,  geometry, 

and  manufacturing  process.  Resources  from  Stress  or  Durability 
and  Damage  Tolerance  analyses  would  be  obtained  as  well  for 
risk  specific  information. 


Step  4a)  The  opportunity  to  apply  advanced  risk  mitigating 
technology  to  the  control  point  is  introduced.  This  is  also  the 
first  step  in  the  iterative  process  which  allows  the  user  to  readjust 
any  inputs,  and  thus,  converge  to  the  ultimate  goal  of  achieving 
an  optimal  strategy. 


Step  4b)  New  technologies  are  researched  and  introduced 
to  the  analysis  with  the  understanding  that,  with  these 
technologies,  each  of  their  capabilities  and  limitations  are 
defined.  Cost  of  the  new  technology(s)  is  investigated  and 
incorporated  into  the  cost/benefit  analysis. 


Step  4c)  Based  upon  user-defined  parameters  and/or  risk 
sensitivity,  control  points  are  selected  for  enhancements.  Some 
locations  may  not  be  appropriate.  For  example,  a  new  composite 
repair  method  may  be  utilized  but  metallic-based  control  points 
are  not  applicable. 

Step  5) A  baseline  system  configuration,  containing  no  new 
or  potential  technology  at  any  control  point,  is  generated. 
Cost/benefit  TPMs  calculated  from  this  baseline  serve  as  a)  a 
reference  for  comparison  and  b)  a  gage  for  optimization  against 
any  other  system  configuration  TPMs.  Modified  configurations 
are  collections  of  control  points  with/without  new  technology. 
These  TPMs  are  also  calculated  and  compared  against  those 
originating  from  the  baseline. 
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Establish  System 
Configuration 


Determine  Which  Control 
Points  are  Technology 
Candidates 


Determine  Parameters  of 
New  Technology 


Perform  Risk  Analysis 


Step  6)  The  structural  risk  analysis  must  be  performed  for 
each  feasible  configuration  using  the  analyses  tools,  parameters 
and  definitions  defined  in  step  1 . 


Perform  Cost  Analysis 


Step  7)  Data  generated  from  the  risk  analysis  are  used  as 
inputs  for  the  cost/benefit  analysis.  Using  the  technical 
performance  measures  (TPMs)  defined  in  step  1,  a  series  of  cost 
calculations  is  performed  for  each  configuration. 


Step  8)  The  TPMs  of  the  configurations  are  compared  via 
the  cost/benefit  analysis  (CBA)  results.  The  goal  is  to  determine 
if  at  least  one  configuration  is  acceptable  from  regulatory  and 
economic  viewpoints.  This  is  the  last  step  in  the  iterative  process 
before  a  viable  configuration  is  selected.  If  no  configuration  is 
deemed  acceptable,  then  step  4a  is  readdressed  with  knowledge 
learned  from  the  previous  iteration  applied  to  the  next 
iteration(s). 


\ 


Select  the  Most  Feasible 
Configuration 


1 


w 


Step  9)  If  the  TPM  results  for  one  (or  a  unique  set  of) 
configuration s)  provide  satisfactory  results,  then  a  decision  is 
made  based  on  a  combination  of  safety  and  cost. 
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4.  Example  Application  of  CBM+SI  Process  Flowchart 

4.1.  Introduction 

To  demonstrate  the  method  described  in  Section  3.2  Flowchart,  a  sixteen  location 
subset  of  the  F-15  Aircraft  C/D  wing  is  analyzed  as  a  complete  system.  A  variety  of 
characteristic  control  points  were  chosen  so  that  the  wing  is  well  represented.  Note  that  the 
selection  of  this  subset  is  fully  explained  in  Section  3.3.  The  focus  in  this  section  is  on  the 
process  which  must  be  followed  in  order  to  obtain  the  information  needed  to  make  an 
informed  decision  regarding  the  maintenance  strategy  and  the  use  of  an  optionally  proposed 
risk  mitigation  technology.  Each  component  of  the  CBM+SI  Process  Flowchart  is  discussed 
here  at  a  high  level.  Later  sections  of  this  report  detail  the  individual  components  at  a  more 
technical  level. 

4.2.  Establish  Analysis  Framework 

4.2.1.  Identify  the  Task 

An  integrated  damage  tolerance  analysis  and  structural  risk  assessment  has  been 
proposed  for  the  F-15  fleet.  The  approach  taken  here  treats  the  inner  and  outer  wing  as  a 
complete  structural  system.  Additionally,  the  application  of  an  in-situ  crack  detection 
technology  is  investigated.  Implementation  of  this  technology  into  the  maintenance 
framework  depends  on  several  factors,  including  safety  of  the  structural  system,  cost,  and 
aircraft  availability  to  perform  missions.  An  analysis  consisting  of  risk  and  cost  analyses 
performed  over  the  entire  life  of  the  structural  system  needs  to  be  performed  in  order  to  make 
an  informed  decision  regarding  the  optimal  maintenance  strategy  for  this  structural  system. 

4.2.2.  Specify  the  Technical  Performance  Measures  &  Analysis  Parameters 

Before  collecting  and  analyzing  data,  the  framework  of  the  analysis  must  be  established 
to  ensure  the  feasibility  of  the  solution  can  be  quantified  and  the  solution  will  be  acceptable 
to  regulatory  authorities. 

4.2.2.I.I.  Technical  Performance  Measures  (TPMs) 

The  cost  of  maintaining  the  fleet  is  summarized  through  the  Net  Present  Value  of 
expenditures  (NPV).  Note  that  this  represents  costs  in  present  day  dollars.  Thus,  a  lower 
NPV  is  preferable.  Note  that  in  finance  generally  the  net  present  value  represents  profit  and 
loss,  however,  in  this  analysis  we  are  concerned  only  with  costs.  Rather  than  place  a  negative 
sign  on  every  value,  costs  are  treated  as  positive.  The  net  present  value  is  actually  the  cost  of 
performing  all  required  maintenance  over  the  life  of  the  fleet  in  today  dollars,  hence,  lower  is 
better. 

In  Phase  I  the  Life  Cycle  Cost  (LCC)  was  also  a  TPM.  LCC  is  similar  to  NPV  but  does 
not  account  for  the  time  value  of  money.  The  LCC  will  be  included  in  more  mature  reports  of 
this  analysis,  but  is  not  discussed  in  this  document. 
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The  availability  of  the  aircraft  is  to  be  summarized  as  Percent  Non-Mission  Capable 
(%NMC).  This  parameter  indicates  the  percentage  of  the  life  for  which  the  aircraft  is 
incapable  of  performing  missions  due  to  downtime  for  scheduled  maintenance,  repairs,  or 
structural  failures.  The  method  for  calculation  of  this  TPM  has  not  yet  been  finalized  in  the 
cost  model.  In  this  document,  the  expected  downtime  for  the  fleet  in  hours  (Fleet  DT)  is 
reported  instead.  This  is  a  simple  measure  that  is  closely  related  to  %NMC. 

4.2.2.1.2.  Risk  Threshold 

The  goal  of  CBM+  is  to  establish  “evidence  of  need”  at  a  time  just  prior  to  performing 
maintenance.  To  fulfill  this  objective,  benchmarks  must  be  established  so  a  comparison 
between  maintenance  strategies,  and  hence  a  judgment,  can  be  performed.  Safety,  cost,  and 
availability  are  the  generalized  parameters  representing  this  study’s  benchmarks.  Cost,  as  a 
TPM,  has  been  discussed  in  the  previous  section.  Safety  and  aircraft  availability  are 
addressed  within  the  Risk  analysis. 

Structural  safety  is  characterized  by  the  risk  associated  with  the  SFPOF  for  each  control 
point.  This  term  is  the  statistical  representation  of  the  likelihood  of  a  part  (or  control  point  in 
this  case)  to  catastrophically  fail  during  flight.  MIL-STD-1530C  documents  the  values  which 
the  SFPOF  must  not  exceed: 

”A  probability  of  catastrophic  failure  at  or  below  10'  per  flight  for  the  aircraft 
structure  is  considered  adequate  to  ensure  safety  for  long-term  military  operations. 
Probabilities  of  catastrophic  failure  exceeding  1  O'5  per  flight  for  the  aircraft  structure  should 
be  considered  unacceptable.  When  the  probability  of  failure  is  between  these  two  limits, 
consideration  should  be  given  to  mitigation  of  risk  through  inspection,  repair,  operational 
restrictions,  modification,  or  replacement.  ” 

The  language  of  MIL-STD-1530C  suggests  that  the  10~7  threshold  applies  to  the 
structural  system.  At  this  stage  of  the  analysis,  this  threshold  is  applied  to  each  control  point 
individually,  rather  than  to  the  system  as  a  whole.  It  is  believed  that  a  thoroughly  performed 
component-level  approach  will  facilitate  a  system-level  approach. 

4.2.2.1.3.  Analysis  Tools 

The  deterministic  damage  tolerance  analysis,  which  must  be  performed  prior  to  the  risk 
analysis,  is  conducted  using  LifeWorks,  a  Boeing  Proprietary  tool  for  Crack  Initiation  and 
Crack  Growth  Analysis.  The  bulk  of  this  analysis  has  already  been  performed  by  the  F-15 
Program,  but  some  re-work  was  requested  during  the  project. 

The  tool  used  to  perform  the  risk  analysis  (calculation  of  SFPOF  and  other  required 
information)  is  the  Boeing  Proprietary  tool  RBDMS  (Risk-Based  Design  and  Maintenance 
System).  The  methodology  of  this  tool  is  discussed  in  the  Risk  Analysis  section  of  this 
report,  Section  6. 

Microsoft  Excel  is  used  to  conduct  the  CBA. 
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4.3.  Acquire  Structural  System  Information 

4.3.1.  Fleet  Parameters 

Several  fleet  characteristics  are  required  for  the  risk  and  cost  analyses.  For  this  analysis 
the  most  relevant  fleet  parameters  are  regarding  the  usage  of  the  aircraft.  The  current 
spectrum,  FTA6,  is  the  most  recent  iteration  of  the  typical  aircraft  usage.  This  spectrum 
represents  a  significant  increase  in  usage  severity  over  the  previous  spectrum  (FSMP).  Other 
usage  parameters  provided  by  the  F-15  program  are  as  follows:  the  typical  platform  flies  300 
flight  hours  (FH)  per  year  at  1.3  FH  per  flight.  The  service  life  for  the  300  aircraft  fleet  is 
assumed  to  be  18,000  FH,  and  this  analysis  is  currently  being  conducted  under  the 
assumption  that  each  platform  is  pristine.  That  is,  this  analysis  is  theoretically  being 
conducted  at  the  beginning  of  the  life  of  the  fleet.  Note  that  the  choice  of  the  18,000  FH 
service  life  will  affect  the  analysis  as  costs  will  be  spread  over  60  years  (300  FH  per  year). 
The  sensitivity  of  the  results  of  this  analysis  to  the  choice  of  lifetime  will  be  investigated  at  a 
later  stage  of  the  project. 

4.3.2.  Maintenance  Parameters 

The  F-15  Program  has  indicated  that,  in  practice,  maintenance  actions  are  generally 
performed  on  flight  hour  multiples  of  200.  For  example,  if  an  NDE  inspection  is  scheduled  to 
occur  at  1116  FH,  it  will  in  actuality  be  performed  at  1200  FH.  To  maintain  as  much  realism 
as  possible,  all  traditional  NDE  inspections  and  repairs  will  take  place  on  200  FH  increments. 

Programmed  Depot  Maintenance  (PDM)  occurs  every  six  calendar  years.  PDM  occurs 
at  the  depot  (as  opposed  to  in  the  field).  At  this  time  the  aircraft  undergoes  significant 
maintenance  on  several  systems  (including  the  structural  system).  Many  locations  in  the  wing 
cannot  be  accessed  in  the  field  due  to  significant  obstruction  of  structure  and  materials. 
Therefore  traditional  NDE  inspections  and  repairs  cannot  take  place  for  certain  locations 
outside  of  PDM. 

In  this  document,  locations  are  distinguished  between  those  easily  accessed  in  the  field 
and  those  which  cannot.  The  easily  accessed  locations  are  referred  to  as  field-accessible.  The 
remaining  locations,  only  accessible  at  PDM,  are  referred  to  as  depot-accessible.  Of  course,  it 
is  implied  that  the  field-accessible  locations  can  also  be  easily  accessed  at  PDM. 

The  accessibility  of  a  location  has  significant  implications  when  contemplating  the 
application  of  in-situ  sensors.  These  sensors  permit  “inspection”  at  any  time  regardless  of 
location.  However,  repairs  cannot  be  reasonably  performed  in  the  field.  In  addition,  all 
technology-assisted  inspection  methods  include  the  possibility  of  false  positives  which 
cannot  be  distinguished  from  true  crack  detections.  Each  time  a  decision  to  repair  is  made  in 
the  field  for  a  location  that  is  impossible  to  access;  the  wing  must  be  disassembled  and 
shipped  to  the  depot  for  repairs.  This  disassembly  is  very  expensive  and  causes  the  aircraft  to 
experience  significant  downtime.  Therefore,  the  implications  of  this  possibility  must  be 
accurately  represented  in  the  cost/benefit  analysis.  These  penalties  are  discussed  in  detail  in 
Section  7  -  Cost  Benefit  Analysis. 
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The  in-situ  inspections  are  assumed  to  take  place  at  either  300  or  600  FH  intervals.  It  is 
required  that  the  intervals  overlap  those  of  the  PDM  cycle  (1800  FH).  The  chance  of  false 
call  occurrences  increases  with  the  frequency  of  inspections,  so  the  risk  analysis  team  is 
using  300  FH  as  an  operating  minimum  and  using  600  FH  intervals  if  possible.  This  will  be 
discussed  in  more  detail  in  Section  4.6. 


4.3.3.  Cost  Parameters 

There  are  numerous  cost  parameters  relevant  for  estimating  the  cost  of  maintaining  a 
fleet  of  aircraft,  such  as  the  hourly  labor  rate  for  maintenance,  the  discount  rate  used  for 
computing  present  and  future  values,  etc.  In  addition,  many  of  the  parameters  pertain  to  the 
installation  and  maintenance  of  an  in-situ  sensor  system.  A  complete  list  of  inputs  required 
by  the  cost  model,  along  with  detailed  explanations,  is  located  in  Section  7. 

4.4.  Acquire  Control  Point  Specific  Information 

The  information  specific  to  each  individual  control  point  required  to  conduct  the  risk 
and  cost  analyses  are  gathered  at  this  stage  of  the  flowchart.  Some  of  the  required 
information  is  shown  below  in  Table  4  to  give  the  reader  a  sense  of  what  information  needs 
to  be  gathered.  The  control  points  are  referred  to  by  Damage  Tolerance  Assessment  (DTA) 
number,  and  organized  in  the  table  according  to  the  accessibility  of  the  location.  This 
accessibility  is  of  paramount  importance  in  determining  a  maintenance  schedule  as  many 
locations  can  only  be  accessed  in  the  depot  when  the  aircraft  is  significantly  disassembled  for 
internal  maintenance.  In  addition  to  that  which  is  shown,  a  damage  tolerance  crack  growth 
analysis  is  required  for  each  control  point.  The  similar  locations  refer  to  the  number  of 
locations  for  which  the  crack  growth  analysis  can  be  thought  to  pertain  to.  For  example,  a 
part  may  possess  10  fastener  holes,  and  each  of  the  two  wings  may  contain  this  part,  so  the 
crack  growth  analysis  for  that  location  refers  to  20  similar  locations. 


Similar 

Small  Crack 

Medium  Crack 

Large  Crack 

Cost  of  Part 

DTA 

Accessibility 

Locations 

Repair  Time 

Repair  Time 

Repair  Time 

Replacement 

055 

Field 

36 

8  (hrs) 

60  (hrs) 

160  (hrs) 

$50,000 

057B 

Field 

32 

8 

84 

160 

60,000 

164 

Field 

154 

60 

60 

240 

100,000 

165 

Field 

32 

60 

60 

240 

100,000 

166B 

Field 

2 

24 

60 

240 

100,000 

184 

Field 

48 

8 

40 

160 

40,000 

187 

Field 

134 

10 

60 

160 

40,000 

188 

Field 

20 

8 

60 

200 

40,000 

097 

Depot 

2 

12 

NA 

200 

100,000 

115 

Depot 

95 

8 

32 

72 

5,000 

124B 

Depot 

12 

8 

60 

160 

50,000 

134B 

Depot 

8 

8 

40 

120 

40,000 

138B 

Depot 

4 

8 

40 

120 

40,000 

143 

Depot 

2 

4 

32 

160 

20,000 

144 

Depot 

2 

24 

40 

60 

5,000 

203 

Depot 

2 

12 

80 

160 

50,000 

Table  4.  Summary  of  Control  Point  Data  for  16  Location  Subset 
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4.5.  Risk  Mitigation  Technology 


4.5.1.  Apply  New  Risk  Mitigation  Technology? 

At  this  point  of  the  flowchart  various  technologies  may  be  considered  for 
implementation  in  the  system.  In  this  analysis,  the  use  of  an  in-situ  crack  detection  system 
which  has  capability  similar  in  scope  to  that  of  traditional  NDE  is  considered. 

4.5.2.  Determine  Parameters  of  New  Technology 

With  any  technology-assisted  crack  detection  system,  there  is  a  trade-off  between  the 
detection  capability  and  the  Probability  of  a  False  Alarm  (henceforth  Pr(FA)).  The  reason  for 
this,  along  with  the  development  of  these  Probability  of  Detection  (POD)  curves  in  general, 
is  detailed  in  Section  5. 

Three  different  POD  curves  pertaining  to  the  in-situ  crack  detection  system  are  utilized 
at  this  stage  of  the  analysis,  each  with  an  associated  Pr(FA).  These  compare  well  with  the 
two  POD  curves  utilized  for  risk  analysis  by  the  F-15  Program,  which  we  are  using  for  NDE 
inspections  on  this  project.  Figure  2  below  depicts  the  five  POD  curves  that  are  used  in  the 
analysis  of  the  16  location  subset,  and  the  median  detection  capability  and  false  call  rate  for 
these  curves  are  shown  in  Table  5.  The  F-15  curves  are  labeled  as  F-15  HF  and  F-15  EF, 
referring  to  the  high  fidelity  and  low  fidelity  NDE  curves,  respectively.  The  in-situ  POD 
curves  are  similarly  named  In-situ  HF,  MF,  and  LF,  for  high,  medium,  and  low  fidelity, 
respectively. 

Interpretation  of  a  POD  curve  is  as  follows:  for  an  existing  crack  of  a  given  size  (found 
along  the  x-axis),  the  curve  gives  the  probability  that  it  will  be  detected  in  a  single  inspection 
(the  y-axis).  The  label  of  the  y-axis,  POD  |  a,  refers  to  the  probability  of  detection  for  a  given 
crack  size,  a.  For  example,  consider  the  Pr(FA)=0.001%  curve  in  Figure  2  (the  red,  long 
dashed  line).  For  this  curve,  a  crack  of  size  0.05”  has  around  a  30%  chance  of  being  found, 
and  a  crack  of  size  0.10”  will  be  detected  in  over  90%  of  inspections. 


Name 

Median 

Detectable 
Crack  (in.) 

False  Call 

Rate 

F-15  HF 

0.025 

? 

F-15  LF 

0.050 

? 

In-situ  HF 

0.039 

1% 

In-situ  MF 

0.044 

0.1% 

In-situ  LF 

0.054 

0.001% 

Table  5.  POD  Curve  Parameters 

The  false  calls  rates  associated  with  the  NDE  inspections  currently  utilized  for 
maintenance  of  the  F-15  fleet  are  not  yet  known.  The  subject  matter  expert  for  determining 
this  has  been  identified  but  has  not  yet  provided  this  information. 
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o 


Crack  Size  (in) 

Figure  2.  Plot  of  POD  Curves  Used  in  Analysis  of  16  Location  Subset 

4.5.3.  Determine  Which  Control  Points  are  Technology  Candidates 

The  team’s  subject  matter  expert  examined  each  control  point  and  determined  that  all 
sixteen  locations  fit  the  criteria  for  the  in-situ  sensor  system  under  consideration.  However, 
several  of  these  locations  are  likely  to  be  low  in  risk,  which  makes  it  doubtful  that  installation 
of  sensors  will  prove  to  be  cost  effective.  That  is,  while  all  locations  are  candidates  for  the 
technology,  several  locations  may  be  dropped  as  candidates  in  either  the  Perform  Risk 
Analysis  or  the  Perform  Cost  Benefit  Analysis  sections  of  the  flowchart. 

4.6.  Determine  System  Configurations 

The  current  maintenance  practice  in  use  in  the  field  is  referred  to  as  the  Baseline 
configuration.  This  includes  NDE  inspections  at  each  location  with  inspection  times  defined 
using  traditional  deterministic  damage  tolerance  methodologies. 

The  Risk-Based  configuration  is  similar  to  the  Baseline.  However,  the  inspection  times 
are  now  defined  by  the  SFPOF.  That  is,  for  each  location,  an  inspection  does  not  occur  until 
the  risk  approaches  the  10~7  threshold.  Note  that  several  of  the  depot-only  locations  in  the 
subset  do  not  have  a  risk-based  scenario  due  to  the  fact  that  they  are  already  inspected  at 
every  PDM  and  the  risk  is  not  sufficiently  low  to  decrease  the  frequency  of  inspections. 


15 

Approved  for  public  release;  distribution  unlimited. 


The  two  configurations  described  above  do  not  include  the  in-situ  sensor  system.  When 
considering  this  technology,  a  large  number  of  configurations  can  exist  as  each  location  may 
or  may  not  have  the  sensors  installed  in  a  given  configuration.  In  addition,  there  are  several 
POD  capabilities  which  can  be  assigned  to  each  location.  Suppose  there  are  3  options  for 
each  location:  no  in-situ  sensors,  in-situ  with  high  fidelity  POD,  and  in-situ  with  low  fidelity 
POD.  If  there  are  16  locations,  there  are  a  total  of  316,  or  more  than  40  million,  possible 
configurations.  Hence  it  is  infeasible  to  perform  the  risk  and  cost  analyses  for  every  possible 
configuration.  The  number  of  acceptable  scenarios  for  each  DTA  item  must  be  limited  in 
order  to  limit  the  total  number  of  potential  configurations  which  must  be  analyzed.  Several 
rules  to  that  effect  are  as  follows: 

•  Field-accessible  locations  use  a  single  in-situ  POD  curve  over  the  life 

o  The  POD  used  for  a  given  location  will  not  change  partway  through  the  life 

•  Depot-accessible  locations  use  at  most  two  in-situ  POD  curves  over  the  life 

o  A  higher  fidelity  system  may  be  used  when  the  aircraft  is  at  the  depot 

■  The  increased  Pr(FA)  is  acceptable  at  the  depot 

o  A  lower  fidelity  system  may  be  used  when  the  aircraft  is  in  the  field 

■  This  is  preferable  since  there  are  large  penalties  associated  with  crack 
detections  and  false  alarms  when  repairs  must  be  performed  in  the 
field 

•  In-situ  inspection  timing  is  limited  to  two  intervals:  300  FH  and  600  FH 

o  These  timings  are  chosen  such  that  the  in-situ  inspections  will  overlap  with 
the  PDM  cycle  (1800  FH  intervals) 

When  a  depot-accessible  location  requires  a  repair  in  the  field,  there  is  a  problem.  The 
location  cannot  be  accessed,  but  the  sensor  system  has  detected  a  crack.  Shipping  the  aircraft 
to  the  depot  for  a  single  repair  (which  may  be  a  false  alarm)  is  infeasible.  After  discussion 
with  the  F- 15  Program,  it  was  determined  that  the  best  option  for  performing  the  repair  is  to 
remove  the  wing  in  the  field,  replace  it  with  a  spare  wing  that  is  kept  on  hand  in  the  field,  and 
ship  only  the  wing  to  the  depot  for  repair.  This  method  carries  with  it  two  penalties  which  are 
accounted  for  in  the  cost  model.  A  repair  penalty  of  500  man  hours  is  added  on  to  the  time 
for  repair,  which  includes  shipping  costs  for  the  wing.  A  downtime  penalty  of  8  hours  is 
applied,  which  reflects  the  time  required  for  a  team  to  remove  and  replace  the  wing.  The  500 
man  hour  penalty  is  severe,  hence  a  method  which  allows  for  the  majority  of  cracks  to  be 
detected  when  the  plane  is  at  the  depot  (where  no  penalties  apply)  would  be  beneficial.  The 
cases  in  which  a  lesser  fidelity  POD  is  used  in  the  field,  and  a  high  fidelity  POD  is  used  at 
the  depot,  are  referred  to  as  Mixed  cases. 

The  available  options  for  each  of  the  16  DTA  items  in  the  subset  are  limited  to  those 
listed  below  in  Table  6.  Note  that  the  mixed  cases  are  only  considered  for  use  with  the  depot- 
accessible  items. 
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Abbreviation 

Inspection  Type 

Pr(FA) 

Inspections 

Base 

Baseline  NDE 

Unknown 

Traditional 

Risk 

Risk-based  NDE 

Unknown 

Risk-based 

HF3 

High  Fidelity  In  Situ 

1.0% 

300  FH 

HF6 

High  Fidelity  In  Situ 

1.0% 

600  FH 

MF3 

Medium  Fidelity  In  Situ 

0.1% 

300  FH 

MF6 

Medium  Fidelity  In  Situ 

0.1% 

600  FH 

LF3 

Low  Fidelity  In  Situ 

0.001% 

300  FH 

LF6 

Low  Fidelity  In  Situ 

0.001% 

600  FH 

MixHL3 

Mix  of  High  &  Low  Fidelity 

1.0%  /  0.001% 

300  FH 

MixHL6 

Mix  of  High  &  Low  Fidelity 

1.0%  /  0.001% 

600  FH 

MixML3 

Mix  of  Medium  &  Low  Fidelity 

0.1%  /  0.001% 

300  FH 

MixML6 

Mix  of  Medium  &  Low  Fidelity 

0.1%  /  0.001% 

600  FH 

Table  6.  Scenario  Options  for  Each  DTA  Item 

A  single  configuration  consists  of  a  selection  of  one  of  the  above  options  for  each  DTA 
in  the  subset.  There  are  far  too  many  potential  configurations  to  consider  them  all. 
Engineering  judgment  must  be  used  to  limit  consideration  to  those  configurations  which  are 
likely  to  be  competitive  when  comparing  TPMs.  That  is,  an  attempt  is  made  to  find  options 
for  each  DTA  item  that  are  obviously  superior  to  the  other  available  options  (lower  risk, 
fewer  required  inspections,  etc).  To  do  this,  RBDMS  is  run  for  each  option  of  each  location. 
Then  the  risk  results  are  compared  and  the  inferior  solutions  are  dropped  from  consideration. 
For  example,  see  the  SFPOF  plots  for  DTA  187  in  Figure  3  below  (only  the  peaks  are  shown 
for  clarity).  Recall,  DTA  187  is  field-accessible,  so  the  mixed  options  are  not  considered. 
Also,  the  high  fidelity  options  are  a  last  resort,  and  they  were  not  required  to  be  run  for  this 
location.  Here,  the  600  FH  interval  options  do  not  sufficiently  reduce  the  risk  and  are 
dropped  from  consideration.  Either  300  FH  interval  option  will  suffice,  hence  they  are  both 
retained. 

This  process  is  carried  out  for  each  DTA  item  and  a  small  number  of  options  are 
retained  for  each  location.  In  Table  7,  the  DTA  items  and  the  available  options  are  placed  in 
a  matrix  and  the  selections  which  are  retained  are  indicated  with  an  “X”. 
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Figure  3.  Peak  SFPOF  Plots  for  DTA  187 


#  to 

permute 

2 

2 

1 

1 

1 

2 

1 

2 

2 

3 

3 

1 

3 

3 

2 

1 


Table  7.  Maintenance  Options  Retained  for  Each  DTA  Item 

The  Base  case  refers  to  the  baseline  configuration  in  which  each  DTA  item  uses  the 
Base  option.  The  SFPOF  of  several  of  the  baseline  scenarios  is  excessively  high.  This  causes 
the  TPM  results  for  the  Baseline  scenario  to  be  unrealistically  high.  This  makes  comparison 
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of  the  other  cases  to  the  baseline  case  difficult  at  this  time.  The  high  risk  issue  is  discussed  in 
Section  6.5. 

Note  that  several  of  the  DTA  items  do  not  have  a  Risk  case.  These  are  depot-accessible 
items  that  are  already  inspected  in  the  baseline  at  every  PDM  and  which  have  risk  levels  that 
are  too  high  to  allow  for  fewer  inspections.  At  this  time,  the  benefit  of  the  Risk  configuration 
can  only  be  approximated  as  those  DTA  items  which  do  not  have  a  Risk  option  use  the  Base 
option  instead. 

The  matrix  shown  above  is  used  to  create  a  complete  permutation  of  all  of  the  in-situ 
configurations  which  will  be  considered.  The  Base  options  are  not  utilized  when  performing 
this  permutation,  hence  the  final  column  in  Table  7  shows  the  number  of  options  which  will 
be  considered  for  each  DTA  item  for  use  in  the  in-situ  configurations.  The  total  number  of 
configurations  to  be  considered  is  the  product  of  this  column:  5184  configurations.  The 
statistical  software  R  is  used  to  perform  this  permutation.  When  also  considering  the  baseline 
and  risk-based  configurations,  a  total  of  5 1 86  configurations  are  run,  of  which  one  or  several 
will  be  recommended  for  use  as  the  system  configuration. 


4.7.  Perform  Risk  Analysis 

Each  checked  box  in  Table  7  corresponds  to  a  single  RBDMS  run.  Hence  there  are  a 
total  of  46  RBDMS  results  files  that  are  to  be  used  in  the  cost  model.  Each  configuration 
consists  of  16  sets  of  RBDMS  results  (one  per  DTA  item).  Section  6  of  this  report  discusses 
the  use  of  RBDMS  for  risk  analysis.  An  example  output  file  is  shown  in  Table  8.  Only  those 
results  which  are  utilized  in  the  CBA  are  shown  here. 


No 

CumFH 

InspType 

PFSLBI 

PFSLAI 

PCDsmall 

PCDmed 

PCDlarge 

0 

0 

0 

. 1776E-14 

. 0000E+00 

. 0000E+00 

. 0000E+00 

. 0000E+00 

1 

7200 

1 

. 8451E-08 

. 12  66E-13 

. 6098E-01 

. 6414E-01 

. 1213E-01 

2 

12600 

1 

. 4988E-08 

. 1510E-13 

. 9438E-01 

. 8552E-01 

. 1469E-01 

3 

18000 

1 

. 5242E-08 

. 4 130E-13 

. 2104E+00 

. 22  61E  +  00 

.  1 677E-01 

Table  8.  Example  RBDMS  Output;  DTA  203  Risk  Option 


The  inspection  number  is  in  the  first  column,  followed  by  the  inspection  times.  The 
SFPOF  before  and  after  inspection  are  labeled  PFSLBI  and  PFSLAI,  respectively.  The  final 
three  columns  give  the  probability  of  detecting  a  crack  of  small,  medium,  or  large  size  at 
each  inspection.  These  are  used  in  the  cost  model  as  larger  cracks  are  much  more  expensive 
to  repair.  Note  that  at  time  zero  (inspection  #  0)  no  inspection  is  performed.  However,  there 
is  an  initial  risk  level  based  on  the  potential  for  flaws  that  are  present  at  time  zero. 


4.8.  Perform  Cost  Benefit  Analysis 

In  this  section  the  costs  associated  with  each  of  the  5186  configurations  for  each  DTA 
are  summarized.  The  mechanics  of  the  cost  benefit  analysis  are  discussed  in  Section  7  of  this 
report.  Note  that  these  results  are  preliminary.  It  is  not  the  intention  of  the  team  that  the 
numbers  presented  here  be  disseminated  to  the  SHM  community  as  evidence  for  or  against 
the  use  of  in-situ  sensors.  Issues  remain  which  are  known  to  skew  the  results.  In  particular, 
several  of  the  Baseline  configurations  exhibit  very  high  risk.  This  causes  the  Baseline  cost 
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estimates  to  hyper-inflate  as  an  unrealistic  number  of  part  failures  is  predicted.  Six  of  the 
locations  in  the  sixteen  location  subset  contain  interference  fit  fasteners,  for  which  a  method 
for  significantly  reducing  the  risk  has  been  determined  but  not  yet  implemented.  This  is 
discussed  in  Section  6.5.1  of  this  report. 

As  described  above,  the  cost  model  is  a  Microsoft  Excel-based  tool.  This  workbook 
performs  the  cost  analysis  for  a  single  configuration.  The  TPMs  for  each  of  the  5184  in-situ 
configurations  are  calculated  by  automating  the  import  of  configuration  parameters  and 
output  of  results  via  a  macro  using  Microsoft’s  programming  language  Visual  Basic  for 
Applications  (VBA).  In  brief,  this  is  performed  by  tabulating  each  configuration  to  be 
examined  and  programmatically  stepping  through  each  configuration  by  importing  the 
appropriate  RBDMS  results,  updating  the  cost  parameters  for  each  DTA  item,  and  tabulating 
the  resulting  TPMs.  This  macro  takes  a  fraction  of  a  second  to  parse  each  configuration  and 
the  process  of  performing  the  calculation  for  all  5184  configurations  takes  roughly  22 
minutes. 

The  in-situ  configurations  with  the  lowest  NPV  (present  value  of  the  costs,  lower  is 
better)  and  the  lowest  total  expected  fleet  downtime  are  shown  below  in  Table  9.  The  fleet 
downtime  is  shown  rather  than  the  %NMC  because  the  method  for  calculating  %NMC  has 
not  yet  been  finalized. 


DTA 

Accessibility 

Min  NPV 

Min  Fleet  DT 

055 

Field 

Risk 

Risk 

057B 

Field 

Risk 

LF6 

164 

Field 

LF3 

LF3 

165 

Field 

LF6 

LF3 

166B 

Field 

Risk 

Risk 

184 

Field 

Risk 

Risk 

187 

Field 

MF3 

MF3 

188 

Field 

LF6 

LF6 

097 

Depot 

Risk 

Risk 

115 

Depot 

MixHL3 

MixHL3 

124B 

Depot 

LF6 

LF6 

134B 

Depot 

MixHL3 

MF3 

138B 

Depot 

MixHL3 

MixHL3 

143 

Depot 

MixML6 

LF6 

144 

Depot 

MixHL3 

MF3 
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Depot 

Risk 

Risk 

Table  9.  In-situ  Configurations  with  Minimum  NPV  and  Fleet  Downtime 


The  NPV  and  expected  fleet  downtime  of  the  baseline,  risk-based,  and  most  preferable 
in-situ  configurations  are  shown  in  Table  10  below.  To  re-iterate,  the  Baseline  configuration 
results  are  inflated  due  to  the  high  risk  of  several  locations  and  are  not  useful  for  comparison 
at  this  time.  Six  of  the  sixteen  locations  will  soon  see  significant  alteration  of  the  risk  as  they 
include  interference  fit  fasteners,  the  effect  of  which  has  not  yet  been  incorporated  in  the 
crack  growth  analysis.  This  alteration  is  discussed  in  Section  6.5.1. 
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Configuration 

NPV  ($b) 

Fleet  DT  (hrs) 

Baseline 

219.209 

7,840,241 

Risk-Based 

3.533 

6,208,154 

In-situ;  Min  NPV 

1.284 

3,732,946 

In-situ;  Min  Fleet  DT 

1.370 

3,676,031 

Table  10.  TPMs  of  the  Baseline,  Risk-based,  and  Preferred  In-situ  Configurations 


For  purposes  of  comparison,  note  that  the  highest  (worst)  NPV  for  an  in-situ 
configuration  was  $1.42  billion,  and  the  maximum  fleet  downtime  for  an  in-situ 
configuration  was  4.9  million  hours. 

4.9.  TPM  Results  Acceptable? 

As  previously  stated,  some  unresolved  issues  remain  which  make  it  difficult  to 
determine  the  benefit  of  the  in-situ  system,  namely  the  high  risks  associated  with  the  baseline 
for  several  locations  and  the  lack  of  a  risk-based  option  for  some  depot-accessible  items. 

If  the  numbers  presented  here  were  thought  at  this  time  to  be  correct,  a  re-design  of 
several  DTA  items  would  be  required  in  order  to  produce  an  acceptable  configuration  that 
does  not  include  the  use  of  an  in-situ  sensor  system.  That  is,  of  the  baseline,  risk-based,  and 
in-situ  configurations,  only  the  in-situ  configuration  is  capable  of  maintaining  the  risk  of  each 
location  below  the  prescribed  threshold.  Hence,  only  the  in-situ  system  configurations  are 
acceptable  according  to  the  framework  described  in  Section  4.2  -  Establish  Analysis 
Framework. 

4.10.  Select  a  Feasible  Configuration 

At  this  stage  of  the  analysis,  the  in-situ  configuration  corresponding  to  the  minimum 
NPV  as  described  in  Table  9  would  be  recommended  for  maintenance  of  the  16  location  F-15 
wing  system.  This  choice  would  be  made  due  to  the  fact  that  the  NPV  is  $80  million  lower 
and  the  fleet  downtime  is  only  1.5%  higher  than  the  minimum  fleet  downtime  configuration. 
If  fleet  availability  were  determined  to  be  the  more  important  parameter,  the  other 
configuration  could  be  selected.  Furthermore,  several  configurations  fall  in  between  these 
values,  thus  one  could  choose  to  carefully  examine  more  candidate  configurations  to  make  a 
decision.  In  this  preliminary  example  these  two  cases  suffice  for  demonstration. 
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5.  In-Situ  Sensor  Capability  Analysis 


The  detection  performance  of  inspection  techniques  is  usually  measured  by  a 
Probability  of  Detection  (POD)  metric  which  is  a  probabilistic  representation,  or  model,  of 
the  likelihood  that  the  inspection  will  find  damage  of  a  certain  size.  As  certain  risk  analysis 
cases  show,  a  benefit  could  be  realized  by  having  the  ability  to  detect  damage  more  often 
than  traditional  ASIP  inspection  cycles.  If  in-situ  automated  technologies  (such  as  SHM)  are 
introduced  to  do  these  inspections  their  performance  will  need  to  be  evaluated  in  terms  of 
metrics  that  can  be  used  to  determine  their  detection  performance  as  well  as  quantify  their 
development  and  operational  costs. 

The  following  is  a  summary  of  the  progress  made  to-date  in  attempting  to  understand 
and  represent  the  detection  capability  of  SHM  technologies  that  could  be  employed  on  the  F- 
15  application  under  consideration.  As  a  starting  point,  it  is  based  on  the  methods  and 
assumptions  defined  in  the  Department  of  Defense  Handbook  on  “Nondestructive  Evaluation 
System  Reliability  Assessment”  (MIL-HDBK-1823A).  Note  that  the  subject  of  POD  for  in- 
situ  detection  systems  is  an  active  research  area  and  is  by  no  means  fully  understood.  To  that 
end,  since  the  risk  analysis  research  in  this  project  is  the  “end  user”  of  the  detection  data  it  is 
critical  that  the  risk  analysis  drive  the  requirements  (and  assumptions)  needed  to  accurately 
model  the  detection  data. 

5.1.  Probability  of  Detection  Approach  of  SHM 

One  necessary  precondition  for  certifying  a  SHM  system  for  flight  is  being  able  to 
provide  a  rigorously  obtained  Probability  of  Detection  (POD)  curve.  The  POD  curve 
provides  information  about  the  smallest  crack  size  that  can  be  reliably  detected  by  an 
inspection  system. 

There  are  two  significant  questions  that  must  be  answered  about  the  performance  of  a 
SHM  system  in  terms  of  POD.  The  first  question  is:  What  is  the  smallest  size  crack  that  can 
reliably  be  detected?  This  crack  size  is  expressed  as  <290/95  where  the  ‘a’  refers  to  the  crack 
length,  ‘90’  refers  to  the  detection  rate,  and  ‘95’  refers  to  the  confidence  level.  The 
confidence  level  is  a  statistical  concept  that  quantifies  the  uncertainty  in  the  estimation. 
Taken  together,  this  means  that  there  is  95%  confidence  that  the  system  will  detect  at  least 
90%  of  the  cracks  of  length  <290/95. 

The  general  process  for  obtaining  these  measures  is  briefly  described  below.  Typically 
one  would  establish  a  relationship  between  an  inspection  system’s  output  (usually  termed  <2) 
and  the  “true”  measured  crack  length,  a.  For  these  studies,  the  relationship  is  established 
between  the  SHM  system’s  Damage  Index  (DI),  which  will  be  called  a,  and  the  measured 
crack  length,  a.  This  relationship  is  established  using  linear  regression  as  follows.  Let  x  = 
f(a)  and  y  =  g(a)  where  /  and  g  are  either  linear  or  logarithmic  functions  selected  such  that  x 
and  y  are  linearly  related.  The  linear  or  logarithmic  representation  of  the  DI,  v,  is  then 
estimated  as: 
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y  -  P o + Ax + e 


(i) 


where  j3o  and  Pi  are  coefficients  to  be  solved  for  and  e  is  the  residual  error  which  is 
normally  distributed  with  a  zero  mean  and  a  variance  8  . 

The  selection  of  an  appropriate  detection  threshold,  yth  ,  is  described  below.  Let  <J>(z) 
be  the  standard  normal  cumulative  distribution  function  and  let  Q( z)  be  the  survivor  function, 
equal  to  l-O(z).  The  POD  function  can  then  be  derived  as: 


POD(a)  =  P(y>y,l)  =  l-Q 


f(a)-u 

=  0 

f(a)-u 

(7 

(7 

where  u  =  and  <y  =  — 


A 


A 


(2) 


This  formula  provides  the  probability  of  detection  for  any  given  crack  size,  a. 
Statistical  techniques  can  be  used  to  calculate  the  95%  confidence  interval  of  this  POD  curve 
and  the  <390/95  point  can  be  selected.  One  option  for  generating  the  POD  curve  is  to  use 
software  based  on  MIL-HDBK-1823A  “Nondestructive  Evaluation  System  Reliability 
Assessment”  available  for  free  download  [4], 


5.2.  False  Call  Rate 

The  second  major  question  involves  determining  the  probability  of  a  false  alarm  (i.e. 
how  often  the  system  indicates  a  crack  when  one  does  not  exist).  The  probability  of  false 
alarm  is  denoted  as  Pr(FA).  Calculation  of  Pr(FA)  is  accomplished  by  collecting  and 
characterizing  a  given  SHM  system  in  the  situation  where  no  crack  exists.  Due  to  in-situ 
environmental  effects,  DI  measurements  taken  when  no  crack  exists  will  have  an  average 
value  greater  than  zero.  This  background  noise  can  be  characterized  by  a  specific  probability 
density  function  (DI  Noise  PDF)  using  statistical  techniques.  This  PDF  allows  the 
calculation  of  the  probability  that  a  given  ‘no  crack’  DI  value  exceeds  the  detection 
threshold  as  shown  in  Figure  4  as  the  shaded  ‘Probability  of  False  Alarm’  area  of  the  DI 
Noise  PDF.  The  shaded  portion  represents  the  proportion  of  times  a  ‘no  crack’  DI  value  will 
exceed  the  detection  threshold.  Similarly,  integrating  the  portion  of  the  DI  scatter  PDF  that 
lies  above  the  detection  threshold  provides  POD(a)  since  that  portion  of  the  function 
represents  the  proportion  of  times  a  DI  for  a  given  size  crack  will  exceed  the  detection 
threshold. 

Both  the  detection  threshold,  yth,  and  Pr(FA)  can  be  calculated  explicitly  as  a  function 
of  one  another.  As  an  example,  assume  the  noise  has  been  analyzed  and  found  to  have  a 
normal  distribution  with  mean  y  and  standard  deviation  a.  For  a  given  probability  of  false 
alarm,  the  detection  threshold  can  be  calculated  as: 


Yth  l^-noise  4"  & noise ^  (1  Pr(FA)) 


(3) 
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Conversely,  the  probability  of  false  alarm  can  be  calculated  as: 

Pr(FA)  =  1-0  fnhZtnoisA  (4) 

V  &  noise  ' 

where  O  represents  the  normal  cumulative  distribution  function. 

As  shown  in  equations  above  and  Figure  4,  background  noise  is  one  of  the  major 
factors  that  determines  false  positive  rates  and  overall  performance  of  a  SHM  system. 
Understanding  key  noise  contributors  and  developing  solutions  to  mitigate  them  are  the  focus 
of  future  design  improvement  and  performance  characterization  efforts. 


Figure  4.  Sample  Pr(FA)  calculation  based  on  background  noise 

5.3.  POD  Curve/False  Call  Examples  from  AFRL  Hot  Spot  Program  using 
MIL-HDBK- 1 823 A  POD  Software 

The  process  of  creating  a  POD  curve  based  on  a  vs.  a-hat  data  can  be  accomplished 
using  POD  software.  Software  based  on  MIL-HDBK- 1823 A  is  available  free  by  download 
from  the  author  (Charles  Annis,  P.E)  who  developed  it  as  part  of  a  DOD  contract  to  write  the 
latest  version  of  MIL-HDBK- 1823 A. 

The  following  pages  will  provide  an  example  of  creating  a  POD  curve  with  data  from  a 
series  of  SHM  experiments  on  six  identical  test  specimens.  Only  three  inputs  are  needed:  a,  a 
and  background  as  defined  in  previous  section. 

Figure  5  shows  the  DI  value  vs.  crack  length  plotted  vs.  various  axes.  Ln  X  and  Ln  Y 
were  selected  as  the  best  linear  representation  of  the  data.  A  linear  relationship  is  needed  to 
have  a  valid  linear  regression  model.  One  consequence  of  having  a  linear  model 
approximate  non-linear  data  is  that  the  properties  of  the  error  of  the  model  would  vary  as  a 
function  of  input  value.  Analysis  done  with  linear  regression  models  assumes  a  constant 
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response,  a  response; 


standard  deviation  of  the  scatter  error.  With  non-linear  data  this  would  not  be  true;  some 
portions  of  the  model  would  have  a  very  high  standard  deviation  and  other  very  low. 


mhl  mhl 


Size,  a  (mils)  Size,  a  (mils) 

Figure  5.  DI  value  (a)  vs.  crack  length  (a)  plotted  on  linear-linear,  log-linear,  linear-log, 

and  log-log  axes 
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Figure  6  shows  the  regression  model  of  the  selected  axis  types  of  the  of  the  system 
response  a  as  a  function  of  crack  size  ‘a’  with  a  95%  confidence  interval.  Note  that  a  log 
scale  was  chosen  for  each  axis.  This  plot  shows  left  (9.805)  and  right  (304.323)  censors 
value  lines.  The  left  censored  was  selected  as  the  smallest  crack  size  in  the  data  set.  The 
right  censored  value  was  selected  as  the  maximum  crack  values  of  the  data  set.  The  decision 
threshold  was  set  to  12.57  mils  which  corresponds  to  a  false  alarm  rate  of  0.1%.  This  will  be 
adjusted  later  on.  The  decision  threshold  is  defined  as  the  value  of  a  above  which  the  signal 
is  interpreted  as  a  hit,  and  below  which  the  signal  is  interpreted  as  a  miss.  It  is  the  a  value 
associated  with  50%  POD. 


Size,  a  (mils)  mhi823 


Figure  6.  A  regression  model  of  ‘a’  vs.  ‘a’  data  from  the  selected  axis  type  with  95% 
confidence  interval  about  line  (the  inner  dashed  lines)  and  scatter  (the  outer  dashed  lines) 


26 

Approved  for  public  release;  distribution  unlimited. 


Theoretical  Normal  CDF  Theoretical  CDF 


Figure  7  shows  noise  data  plotted  on  four  different  types  of  probability  ‘paper’.  The 
natural  log  distribution  appears  to  be  the  best  background  noise  model.  Figure  8  shows  a 
magnified  version  of  Figure  7d  showing  a  close-up  of  the  probability  distribution  plot.  It 
represents  the  selected  form  of  the  distribution  (for  later  analysis)  and  provides  values  for  the 
distributions  parameters. 


Weibull 


lo9e(a  noise) 


exponential  |0ge(£  noise) 


Gaussian 


3  noise 


mh1823 


lognormal 


3  noise 


mh1823 


Figure  7.  Noise  data  plotted  on  four  different  types  of  probability  ‘paper’ 
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Theoretical  Normal  CDF 


ICQ  (a  noise) 


mh1823 


Figure  8.  The  selected  the  probability  distribution  plot  for  SHM  background  noise 

Figure  9  shows  a  trade-off  plot  of  getting  false  positives  and  <290/95  as  a  function  of 
selected  <2  decision.  As  shown  in  Figure  9,  changing  the  decision  threshold  changes  both  the 
probability  of  false  positive  and  the  critical  target  sizes  <250  (050  is  a  decision),  aw  and  <290/95. 


Using  this  plot,  three  different  probability  of  false  positive  are  considered: 


•  False  alarm  rate  of  .001  corresponds  to  a  threshold:  12.57 

•  False  alarm  rate  of  .01  corresponds  to  a  threshold:  10.01 

•  False  alarm  rate  of  .05  corresponds  to  a  threshold:  8. 166 
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Figure  9.  A  trade-off  plot  showing  the  probability  of  getting  false  positives  and  090/95  as  a 

function  of  selected  a  decision 
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Probability  of  False  Positive 


response, 


Results  for  False  Alarm  Rate  of  .001  (0.1%): 


Figure  10.  The  regression  model  of  Figure  6  superimposed  with:  a  POD  curve,  the  noise 
distribution,  and  probability  of  detection  distributions  based  on  prediction  scatter  placed 

at  various  detection  points 
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response, 


Results  for  False  Alarm  Rate  of  .01  (1%): 


Size,  a  (mils)  mhi823 


Figure  12.  The  regression  model  of  Figure  6  superimposed  with:  a  POD  curve,  the  noise 
distribution,  and  probability  of  detection  distributions  based  on  prediction  scatter  placed 

at  various  detection  points 
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response, 


Results  for  False  Alarm  Rate  of  .05  (5%): 


Size,  a  (mils)  mhi823 


Figure  14.  The  regression  model  of  Figure  6  superimposed  with:  a  POD  curve,  the  noise 
distribution,  and  probability  of  detection  distributions  based  on  prediction  scatter  placed 

at  various  detection  points 
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Summary  of  Log  -  Log  Results: 

•  False  alarm  rate  of  .001  corresponds  to  a  threshold:  12.57,  <290/95  =  89.26 

•  False  alarm  rate  of  .01  corresponds  to  a  threshold:  10.01,  <290/95  =  80.19 

•  False  alarm  rate  of  .05  corresponds  to  a  threshold:  8.166,  <290/95  =  72.94 
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6.  Risk  Analysis  Progress 

6.1.  RBDMS 

6.1.1.  Overview  of  RBDMS 

The  Reliability-Based  Design  and  Maintenance  System  (RBDMS)  and  the  PRObability 
of  Fracture  (PROF)  were  developed  to  estimate  the  probability  of  failure  for  damage  tolerant 
structure.  The  PROF  code  was  developed  by  the  University  of  Dayton  Research  Institute  and 
the  RBDMS  code  was  developed  by  The  Boeing  Company.  Both  codes  use  the  same 
analysis  process  as  shown  in  Figure  16.  With  all  the  input  parameters  defined,  both  codes  are 
able  to  solve  and  produce  the  SFPOF  and  the  percentage  of  crack  detected  output  data  for  the 
cost  analysis.  However,  since  both  codes  use  different  probabilistic  analysis  strategies  and 
different  ways  to  manage  the  statistical  distributions,  their  calculated  results  can  differ. 


Figure  16:  Risk  Analysis  Input  Parameters  and  Single  Flight  Probability  of  Failure 

(SFPOF)[l] 

6.1.2.  RBDMS  Code  Modifications 

In  the  following,  the  recent  RBDMS  code  modifications  are  briefly  discussed.  Details 
of  these  modifications  can  be  reviewed  in  Appendix  A.3. 

•  SFPOF  system  reliability  calculation  formula  is  modified 

o  Properly  calculates  the  risk  for  multiple  similar  locations 
o  Corrected  an  error  which  would  occur  if  the  EIFS  is  too  large  in  comparison 
to  the  critical  crack  length  (a  warning  is  now  issued  when  this  is  the  case) 

•  Two  inspection  types  (i.e.,  multiple  PODs)  can  be  used  in  a  single  run 

•  For  each  inspection  the  Probability  of  Crack  Detected  is  subdivided  into  small, 
medium,  and  large  crack  zones 
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o  The  user  provides  two  crack  size  thresholds  to  segregate  the  zones 
o  Required  by  the  cost  model  since  larger  cracks  are  more  expensive  to  repair 

•  Time  zero  SFPOF  calculation  has  been  implemented  so  that  SFPOF  plots  realistically 
represent  the  fact  that  the  initial  SFPOF  is  non-zero  due  to  the  existence  of  an  initial 
flaw 

•  A  tabular  summary  output  fde  is  generated  for  easy  import  into  the  cost  model 

•  The  output  file  names  have  been  made  more  user  friendly 

6.2.  Sixteen  Location  Subset 

6.2.1.  Selection 

There  are  several  dozen  Aircraft  C/D  wing  locations.  The  data  for  these  were  provided 
by  the  F-15  Program  at  a  rate  of  several  locations  per  week.  While  this  information  was 
being  prepared  it  was  determined  that  the  analysis  should  begin  on  a  subset  of  the  full  list  so 
that  work  could  begin.  Sixteen  locations  were  chosen,  including  items  exhibiting  a  range  of 
risks.  Also,  half  of  the  items  are  easily  accessed,  and  the  remainder  are  inaccessible  in  the 
field.  As  stated  in  Section  4,  items  that  can  be  easily  accessed  at  any  time  are  referred  to  as 
field-accessible,  and  locations  that  can  only  be  accessed  at  the  depot  are  referred  to  as  depot- 
accessible. 


6.2.2.  Brief  Description 

The  locations  of  the  subset  are  identified  below  in  Table  11.  As  a  measure  of  the 
relative  risk  of  each  location,  the  time  until  the  10"7  risk  threshold  is  breached  with  no 
inspections  is  shown.  Note  there  is  a  large  range  of  risks  represented  in  the  subset,  with  the 
threshold  being  breached  from  a  range  of  200  to  16,200  FH. 


FH  Breach  Similar 


Accessibility 

DTA 

Description 

Threshold 

Locations 

055 

Front  spar  lower  flange,  XW  169.5 

1600 

36 

057B 

Rear  spar  lower  flange,  XW  167.3 

3000 

32 

164 

Lower  wing  skin  at  rear  spar,  XW  79 

5300 

154 

Field 

165 

Lower  wing  skin  at  shoulder  rib  &  intermediate  spar,  fastener  hole 

5900 

32 

166B 

Lower  wing  skin  at  shoulder  rib  and  intermediate  spar,  bend  radius 

16200 

2 

184 

Lower  fwd  wing  skin  at  XW  188 

1100 

48 

187 

Lower  forward  wing  skin  surface  cracks  at  front  spar 

700 

134 

188 

Lower  aft  wing  skin  at  main  spar,  XW  163.6 

600 

20 

097 

Intermediate  spar  seal  groove  at  lower  lug  backup 

9800 

2 

115 

Aft  lower  skin 

200 

95 

124B 

Front  spar  lower  flange  at  XW  224.8 

3800 

12 

Depot 

134B 

T.E.  closure  spar  -  upper  flange 

1200 

8 

138B 

T.E.  closure  spar  xw  224 

1200 

4 

143 

Wing  fairing  side  panel  rib  cap 

1800 

2 

144 

Inboard  T.E.  rib  at  XW  108.83 

700 

2 

203 

Front  spar  tooling  hole  at  XW  216 

7600 

2 

Table  11.  Description  of  the  Sixteen  Location  Subset 
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6.3.  Inspection  Scenarios  for  Each  DTA 

As  indicated  in  Section  4.6,  there  are  several  inspection  scenarios  to  consider  for  each 
DTA  item.  These  are  further  described  as  follows: 

•  Baseline 

o  NDE  inspections  only 

o  Inspection  times  determined  by  traditional  deterministic  damage  tolerance  with 
rule-of-thumb  knockdown  factors 

•  Risk-based 

o  NDE  inspections  only 

o  Inspection  times  determined  by  component-level  SFPOF  threshold  of  10'7 

•  In-situ 

o  Locations  without  in-situ  sensors 

■  Results  correspond  to  the  risk-based  NDE  scenario  for  these  items 
o  Field-accessible  locations  with  in-situ  sensors 

■  No  penalty  for  crack  detections  made  in  the  field 

■  Single  in-situ  POD  curve  used  over  the  life  of  the  platform 
o  Depot-accessible  locations  with  in-situ  sensors 

■  500  man  hour  penalty  and  8  hour  downtime  penalty  for  repairs  made  in 
the  field 

■  Field  POD  curve  may  be  of  low  fidelity 

■  Depot  POD  curve  may  be  of  high  fidelity 

6.3.1.  Baseline  Inspection  Times 

The  first  step  in  performing  the  risk  analysis  for  the  baseline  case  is  the  determination 
of  the  inspection  times  that  are  used  in  the  field.  Note  that  the  determination  of  baseline  times 
differs  between  field-accessible  and  depot-accessible  locations,  as  NDE  inspections  can  only 
occur  for  depot-accessible  locations  at  multiples  of  1800  FH.  That  is,  PDM  occurs  only  at 
flight  hours  1800,  3600,  5200,  7200,  9000,  10800,  12600,  14400,  16200,  and  18000,  so 
depot-accessible  locations  can  only  be  inspected  at  these  times.  Several  characteristic 
example  calculations  are  shown  in  this  section. 

6.3. 1.1.1.  Field-Accessible  Location:  DTA  166B 

The  relevant  parameters  for  determining  the  baseline  inspection  times  follow.  These 
parameters  were  obtained  from  the  F- 15  Program. 

•  Safety  Limit 

o  The  life  predicted  by  the  deterministic  damage  tolerance  analysis 

•  DI 

o  Multiplied  by  the  safety  limit  to  determine  the  first  inspection  time 

•  ADI 

o  Multiplied  by  the  safety  limit  to  determine  the  subsequent  inspection  times 


38 

Approved  for  public  release;  distribution  unlimited. 


For  DTA  166B,  the  safety  limit,  DI,  and  ADI  are  3500  FH,  0.5,  and  0.5,  respectively. 
Note  that  the  DI  and  ADI  are  not  necessarily  equal  in  general. 

Recall  that  NDE  inspections  in  this  analysis  will  only  occur  at  multiples  of  200  FH.  To 
more  accurately  represent  what  is  done  in  the  field,  the  inspection  times  up  to  18,000  FH  are 
first  calculated  and  then  the  times  are  rounded  to  the  nearest  200  FH  multiple.  Note  that  the 
final  inspection  time  was  always  cut  off  at  18,000  FH.  See  Table  12  below. 

Inspection#  1  2  3  4  5  6  7  8  91011 

Interval  Time  1750  1750  1750  1750  1750  1750  1750  1750  1750  1750  1750 

Cumulative  Time  1750  3500  5250  7000  8750  10500  12250  14000  15750  17500  19250 

Baseline  Inspection  Times  1800  3600  5200  7000  8800  10600  12200  14000  15800  17600  18000 


Table  12.  DTA  166B  Baseline  Inspection  Times 


6.3.1. 1.2.  Depot-Accessible  Location:  DTA097 

Depot-accessible  locations  can  only  be  inspected  at  PDM.  The  maintenance  plan 
occasionally  specifies  a  safety  limit,  DI,  and  ADI  as  in  the  case  of  field-accessible  locations. 
The  times  are  calculated  in  similar  fashion.  However,  rounding  is  somewhat  different.  The 
inspection  time  is  rounded  to  the  earlier  1800  FH  multiple,  unless  it  is  within  400  FH  of  the 
next  1800  FH  multiple.  As  an  example  DTA  097  is  calculated  as  follows.  The  safety  limit, 
DI,  and  ADI  are  9000  FH,  0.5,  and  0.5,  respectively.  See  Table  13  below. 

Note  that  for  a  location  such  as  this,  it  is  possible  to  perform  NDE  inspections  more 
often  if  desired,  since  there  are  currently  PDMs  at  which  this  location  is  not  inspected. 


Inspection  # 

Interval  Time 
Cumulative  Time 
Baseline  Inspection  Times 


12  3  4 
4500  4500  4500  4500 
4500  9000  13500  18000 
3600  9000  12600  18000 


Table  13.  DTA  097  Baseline  Inspection  Times 


6.3. 1.1.3.  Depot-Accessible  Location:  DTA124B 

When  the  maintenance  plan  for  a  depot-accessible  location  does  not  indicate  a  value  for 
DI,  and  ADI,  it  instead  calls  for  inspection  at  every  PDM.  For  these  locations,  the  baseline 
inspection  times  are  simply  the  multiples  of  1800  up  to  18000  FH,  or:  1800,  3600,  5200, 
7200,  9000,  10800,  12600,  14400,  16200,  and  18000.  This  is  the  case  for  DTA  124B. 

Note  that  for  each  of  these  locations,  performing  more  NDE  inspections  is  not  possible. 
The  location  is  already  inspected  at  every  PDM.  Thus  the  risk-based  scenario  is  equivalent  to 
the  baseline  scenario. 
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6.3.2.  Risk-Based  Inspection  Times 

For  convenience  the  risk-based  maintenance  strategy  is  referred  to  as  the  Risk  strategy. 
In  this  scenario,  the  inspection  times  are  determined  by  the  RBDMS  results.  Each  inspection 
is  scheduled  at  the  200  FH  multiple  nearest  the  flight  hour  where  the  SFPOF  of  a  location 
reaches  the  10'7  threshold,  allowing  for  slight  exceedance  at  this  stage  of  the  analysis.  Several 
characteristic  examples  are  shown  here. 

The  SFPOF  plots  for  the  baseline  and  risk  scenarios  for  DTA  166B  (field-accessible) 
are  shown  in  Figure  17.  In  this  case,  the  baseline  is  very  conservative.  In  the  risk  scenario 
only  two  inspections  are  required  to  keep  the  SFPOF  below  the  10'7  threshold,  whereas  the 
baseline  scenario  includes  eleven  inspections  (the  first  several  inspections  do  not  alter  the 
SFPOF  as  the  risk  is  too  low  to  see  on  the  plot). 


O 

CL 

LL 
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5000  10000 

Flight  Hours 


15000 


Figure  17.  DTA  166B  SFPOF  Plots  for  the  Baseline  and  Risk  Scenarios 


The  SFPOF  plot  for  the  baseline  scenario  of  the  depot-accessible  location  DTA  124B  is 
shown  below  in  Figure  18.  Recall  that  in  the  baseline  scenario  there  is  an  NDE  inspection  at 
every  PDM.  There  is  no  risk  scenario  for  this  location  due  to  the  fact  that  the  risk  exceeds  the 
threshold  when  the  location  is  inspected  at  every  opportunity.  For  this  location,  the  only 
options  for  reducing  the  risk  to  an  acceptable  level  are  to  re-design  the  part  or  to  utilize  in- 
situ  sensors  to  inspect  the  location  more  often.  This  is  explored  in  the  next  section. 
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Figure  18.  DTA  124B  SFPOF  Plot  for  the  Baseline  Scenario 


In  Figure  19  below  are  the  SFPOF  plots  for  DTA  055  (field-accessible).  Note  that  the 
risk  of  the  baseline  scenario,  with  only  six  inspections,  is  excessively  high.  This  high  risk 
issue  is  discussed  in  Section  6.5.  For  this  location,  34  NDE  inspections  -  with  a 
correspondingly  high  number  of  man-hours  -  are  required  over  the  life  to  remain  beneath  the 
risk  threshold.  Hence  this  location  is  an  excellent  candidate  for  in-situ  sensors. 
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Figure  19.  DTA  055  SFPOF  Plots  for  the  Baseline  and  Risk  Scenarios 
6.3.3.  In-Situ  Inspection  Times 

As  stated  previously,  the  in-situ  scenarios  will  have  constant  inspection  intervals  of 
either  300  or  600  FH  depending  on  the  location.  If  the  600  FH  interval  will  suffice,  it  is 
preferable  to  the  300  FH  interval  due  to  the  fact  that  half  as  many  false  calls  are  expected  to 
occur  over  the  life  (this  is  particularly  relevant  if  the  POD  curve  being  used  is  of  the  medium 
or  high  fidelity  variety).  Several  examples  of  the  selection  of  POD  capability  and  inspection 
frequency  are  shown  here,  DTA  184  (field-accessible),  DTA  164  (field-accessible)  and  DTA 
138B  (depot-accessible).  Recall  the  list  of  acronyms  for  the  strategies  under  consideration  as 
shown  in  Table  6  on  page  17. 

An  inspection  interval  of  600  FH  cannot  be  used  for  DTA  184  because  the  risk  exceeds 
the  threshold  at  several  points.  SFPOF  plots  are  shown  in  Figure  20  with  inspections  at  300 
FH  intervals  utilizing  the  medium  and  low  fidelity  POD  curves.  The  risk  of  each  of  these 
strategies  is  similar;  therefore  both  are  retained  as  potential  strategies.  The  high  fidelity  POD 
curve  is  not  considered  unless  it  is  absolutely  required  to  maintain  the  risk  at  acceptable 
levels,  as  the  false  call  rate  is  relatively  high. 
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Figure  20.  DTA  184  SFPOF  Plots  for  the  In-situ  Scenario 


In  Figure  21,  the  SFPOF  of  DTA  164  using  the  low  fidelity  POD  for  both  300  and  600 
FH  intervals  are  shown  (LF3  and  LF6).  The  low  fidelity  POD  corresponds  to  a  false  call  rate 
of  0.001%  per  inspection.  The  lesser  fidelity  POD  curve  is  used  for  this  field-accessible 
location  because  DTA  164  includes  154  similar  locations  and  the  small  crack  repair  time  for 
this  location  is  60  man-hours,  thus  the  penalty  for  a  false  alarm  is  a  concern  as  the 
consequence  for  a  false  alarm  is  an  unnecessary  small  crack  repair.  For  this  location,  the 
interval  of  600  FH  may  be  more  appropriate  as  the  risk  is  adequately  controlled  and 
increasing  the  inspection  frequency  to  every  300  FH  will  double  the  false  call  rate.  However, 
the  choice  between  these  two  options  cannot  be  made  with  the  risk  analysis  alone.  The 
increased  risk  of  the  600  FH  intervals  may  result  in  a  higher  cost  due  to  the  increased  chance 
of  finding  large  cracks  at  later  inspections.  Hence  both  of  these  options  must  be  analyzed 
using  the  cost  model  to  determine  the  optimal  choice,  and  both  are  included  in  the  grid 
shown  in  Table  7. 
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Figure  21.  DTA  164  SFPOF  Plot  for  the  In-situ  Scenario 


Figure  22  displays  the  Probability  of  Crack  Detection  (PCD)  of  medium  and  large  size 
at  each  inspection  for  DTA  164.  With  600  FH  inspection  intervals,  the  chance  of  finding 
medium  cracks  is  increased,  and  there  is  now  a  chance  that  large  cracks  will  be  found  later  on 
in  the  life.  The  repair  times  for  medium  and  large  cracks  are  60  and  240  man-hours, 
respectively.  In  addition,  if  a  large  crack  is  found  there  is  a  $100,000  part  replacement  cost. 
Hence  it  is  possible  that  the  chance  of  finding  larger  cracks  will  outweigh  the  costs 
associated  with  false  calls.  It  is  not  possible  to  choose  the  optimal  strategy  based  on  the  risk 
analysis  alone.  The  cost  model  is  required  to  make  the  optimal  selection;  therefore  both 
options  are  retained  for  consideration. 
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Figure  22.  Probability  of  Detecting  Cracks  of  Medium  and  Large  Size,  DTA  164 


Each  of  the  DTA  items  discussed  above  are  field-accessible,  therefore  only  one  POD 
curve  is  used  over  the  life  of  the  platform.  For  depot-accessible  locations,  it  may  be 
preferable  to  use  a  high  fidelity  POD  curve  when  the  aircraft  is  at  the  depot  so  that  as  many 
cracks  as  possible  are  found  at  these  times.  When  away  from  the  depot,  a  lesser  fidelity  POD 
curve  is  used  to  limit  crack  detections  and  therefore  limit  the  number  of  times  the  wing  will 
need  to  be  detached  and  shipped  to  the  depot  for  repairs,  thus  incurring  large  penalties  in  the 
cost  model. 

This  scenario  is  examined  here  for  DTA  138B,  a  depot-accessible  location.  Figure  23  is 
a  combined  plot  of  the  SFPOF  and  the  PCD  of  large  cracks  for  this  location  with  strategy 
LF3  throughout  the  life.  Recall,  the  low  fidelity  POD  is  used  because  there  are  significant 
cost  penalties  for  depot-accessible  DTA  items  when  cracks  are  detected  away  from  the  depot. 
The  risk  is  maintained  at  the  threshold  using  this  POD;  however,  the  in-situ  inspections 
which  occur  at  the  depot  possess  unnecessarily  low  capability  as  the  penalties  do  not  apply 
for  these  inspections. 
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Figure  23.  DTA  138B  Combined  SFPOF  and  PCD  Plot  with  Low  Fidelity  POD 


In  Figure  24  below,  the  capability  of  the  in-situ  sensors  differs  between  the  depot  and 
the  field.  The  strategy  used  is  MixHL3.  That  is,  in  the  depot,  the  high  fidelity  POD  with 
Pr(FA)  =  1%  is  used  with  the  hope  of  finding  all  cracks  before  the  plane  is  sent  out  into  the 
field.  In  the  field,  the  low  fidelity  POD  with  Pr(FA)  =  0.001%  is  used  with  the  intention  of 
only  discovering  larger  cracks.  Note  that  the  chance  of  discovering  large  cracks  has  been 
significantly  reduced  in  general  without  much  change  in  the  SFPOF.  The  other  options  for 
this  DTA  item  are  not  shown.  This  strategy  appears  to  be  superior  to  the  others,  therefore  it  is 
the  only  in-situ  strategy  retained  in  the  grid  of  Table  7. 
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Figure  24.  DTA  138B  Combined  SFPOF  and  PCD  Plot  with  Differentiated  Fidelity  POD 
6.4.  RBDMS  Results  Summary 

The  complete  details  of  every  risk  analysis  included  for  consideration  in  the  system  are 
not  shown  in  this  document.  A  summary  of  these  risk  runs  is  shown  below  in  Table  14. 
Descriptions  for  the  columns  precede  the  table.  Note  that  when  implementing  the  Risk 
strategy  for  this  preliminary  analysis  the  10"7  threshold  was  aimed  for,  hence  many  of  the 
risk-based  SFPOF  numbers  are  very  close  to  10~7  (e.g.  1.05*1 0'7).  The  column  which  counts 
those  risks  over  the  threshold  used  2*10'7  instead  so  as  not  to  count  these  borderline  cases. 

•  Maximum  SFPOF 

o  The  highest  SFPOF  at  any  time 

•  Median  SFPOF 

o  The  median  value  of  SFPOF  before  inspections  (the  peaks  only) 

•  Maximum  Large  PCD 

o  The  maximum  of  the  probabilities  of  detecting  a  large  crack  at  inspection 

•  Number  Insp’s 

o  The  number  of  inspections  that  take  place  over  the  1 8,000  flight  hour  life 

•  Number  >  2  *  1  O'7 

o  The  number  of  inspections  at  which  the  SFPOF  is  greater  than  2*1  O'7 

•  Number  >1 0'5 

o  The  number  of  inspections  at  which  the  SFPOF  is  greater  than  10'5 
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Maximum 

Median 

Maximum 

Number 

Number 

Number 

DTA 

Strategy 

SFPOF 

SFPOF 

Large  PCD 

Insp's 

>  2E-7 

>  IE-5 

dta055 

Base 

1.0E+00 

7.2E-02 

1.5E-03 

6 

6 

6 

LF3 

2.8E-12 

5.7E-13 

0.0E+00 

60 

0 

0 

Risk 

1.4E-07 

4.8E-08 

9.8E-10 

34 

0 

0 

dta057B 

Base 

1.0E+00 

5.0E-01 

2.0E-01 

2 

2 

2 

LF6 

1.2E-13 

6.0E-14 

0.0E+00 

30 

0 

0 

Risk 

1.1E-07 

6.4E-08 

6.7E-09 

16 

0 

0 

dta097 

Base 

3. IE- 15 

3. IE- 15 

1.0E+00 

5 

0 

0 

Risk 

2.7E-08 

1.7E-08 

1.0E+00 

2 

0 

0 

dtall5 

Base 

9.5E-01 

6.0E-01 

0.0E+00 

10 

10 

10 

MixHL3 

6.7E-03 

9.0E-04 

0.0E+00 

60 

60 

60 

dtal24B 

Base 

7.4E-06 

1.5E-06 

8.6E-04 

10 

7 

0 

LF6 

7.5E-11 

1.1E-11 

5.6E-12 

30 

0 

0 

dtal34B 

Base 

4.2E-04 

3.1E-04 

2.6E-01 

10 

10 

10 

MF3 

7.6E-07 

2.5E-07 

3.8E-09 

60 

48 

0 

MixHL3 

2.8E-06 

5.5E-07 

9.1E-08 

60 

49 

0 

dtal38B 

Base 

2.1E-03 

8.9E-04 

9.4E-04 

10 

10 

9 

MixHL3 

1.9E-07 

1.3E-07 

1.9E-15 

60 

0 

0 

dtal43 

Base 

2.3E-04 

1.2E-04 

5.7E-03 

10 

9 

9 

LF6 

7.9E-14 

3.3E-14 

3.2E-06 

30 

0 

0 

MixML6 

1.5E-14 

8.4E-15 

6.1E-07 

30 

0 

0 

dtal44 

Base 

5.4E-02 

1.7E-02 

6.6E-03 

10 

10 

10 

MF3 

3.4E-07 

7.6E-08 

2.2E-12 

60 

4 

0 

MixHL3 

1.0E-06 

1.9E-07 

2.3E-10 

60 

29 

0 

dtal64 

Base 

1.0E+00 

5.7E-05 

9.1E-07 

5 

5 

4 

LF3 

2.9E-13 

3.4E-14 

0.0E+00 

60 

0 

0 

LF6 

2. IE- 10 

3.4E-12 

7.5E-14 

30 

0 

0 

Risk 

1.1E-07 

5.2E-08 

1.1E-09 

20 

0 

0 

dtal65 

Base 

7.3E-01 

8.4E-05 

4.9E-06 

8 

8 

7 

LF3 

2. IE- 13 

4.6E-14 

0.0E+00 

60 

0 

0 

LF6 

5.8E-08 

4.5E-09 

9.6E-10 

30 

0 

0 

Risk 

8.9E-08 

2.9E-08 

4.4E-09 

26 

0 

0 

dtal66B 

Base 

2.6E-12 

4.4E-16 

0.0E+00 

11 

0 

0 

Risk 

1.1E-07 

8.5E-09 

2.8E-08 

2 

0 

0 

dtal84 

Base 

3.9E-05 

7.3E-08 

1.1E-09 

30 

1 

1 

LF3 

1.7E-08 

7.4E-09 

2.5E-13 

60 

0 

0 

MF3 

8.7E-09 

4.0E-09 

3.0E-14 

60 

0 

0 

Risk 

1.3E-07 

7.7E-08 

1.2E-09 

31 

0 

0 

dtal87 

Base 

9.2E-04 

6.1E-08 

0.0E+00 

30 

2 

1 

LF3 

2.3E-08 

2.7E-09 

0.0E+00 

60 

0 

0 

MF3 

2.3E-08 

1.5E-10 

0.0E+00 

60 

0 

0 

Risk 

1.9E-07 

5.8E-08 

0.0E+00 

31 

0 

0 

dtal88 

Base 

1.4E-03 

1.1E-04 

5.5E-06 

4 

4 

4 

LF6 

1.9E-11 

6.2E-12 

1.5E-16 

30 

0 

0 

Risk 

1.2E-07 

1.0E-07 

5.4E-09 

9 

0 

0 

dta203 

Base 

5.2E-12 

2.3E-12 

5.4E-07 

10 

0 

0 

Risk 

8.5E-09 

5.1E-09 

1.3E-02 

3 

0 

0 

Table  14.  RBDMS  Results  Summary 
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Several  locations  in  this  table  have  very  high  risk  and  exceed  the  threshold  repeatedly. 
This  issue  is  discussed  in  the  next  subsection. 

6.5.  High  Risk  Locations 

The  SFPOF  of  several  locations  in  the  sixteen  location  subset  is  excessively  high.  Some 
of  these  were  shown  earlier  in  this  section.  An  additional  example,  DTA  115,  is  shown  in 
Figure  25  below.  The  MixHL3  is  the  best  strategy;  hence  it  is  used  in  this  preliminary 
analysis.  However,  it  is  not  acceptable. 


O 

Q. 

LU 

l/) 


- IE-5 


IE-7 


MixHL3 
—  —  —  Base 


0  5000  10000  15000 

Flight  Hours 


Figure  25.  Baseline  and  Mixed  Options  for  DTA  115 

A  number  of  methods  for  mitigating  the  high  risks  were  attempted  with  mixed  success. 
These  are  discussed  in  this  section.  Note  that  for  several  locations,  such  as  DTA  115,  the 
issue  is  not  yet  resolved. 

6.5.1.  Residual  Stresses 

It  was  identified  that  several  of  these  locations  are  either  cold-worked  holes  or  are 
fastener  holes  in  which  interference  fit  fasteners  are  installed.  Each  of  these  methods 
introduces  residual  stresses  in  the  control  point  that  increase  the  fatigue  life.  The  F-15 
Program  has  traditionally  represented  these  residual  stresses  in  a  damage  tolerance  analysis 
by  reducing  the  initial  flaw  size  to  0.003”  from  the  usual  rogue  flaw  size  of  0.05”.  This 
significantly  increases  the  life  by  providing  a  period  of  time  before  the  crack  reaches  the 
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previous  initial  size  of  0.05”.  This  increases  the  flight  hour  at  which  the  critical  crack  size  is 
reached. 

However,  in  an  RBDMS  analysis,  the  final  flight  hour  associated  with  the  critical  crack 
size  is  not  relevant.  Rather,  the  crack  growth  curve  is  used  to  determine  how  quickly  to  grow 
the  EIFS  distribution.  That  is,  a  crack  of  size  0.05”  in  RBDMS  will  grow  at  the  same  rate 
regardless  of  the  initial  flaw  size  in  the  crack  growth  table.  The  initial  crack  size  workaround 
is  an  insufficient  method  for  representing  the  residual  stresses  in  the  RBDMS  analysis. 

The  F-15  Program  has  been  working  with  the  AFRL  software  PROF  in  recent  years  and 
has  developed  a  method  for  incorporating  residual  stresses  in  the  damage  tolerance  analysis 
in  a  fundamental  way.  The  technical  details  of  this  method  are  located  in  Appendix  A.2. 

This  method  was  used  to  decrease  the  criticality  of  DTA  188.  Figure  26  below  shows 
the  baseline  risk  before  and  after  this  method  was  used.  Even  though  the  risk  remains 
relatively  high,  clearly  this  method  is  very  useful  for  reducing  the  excessive  risks  for  those 
locations  for  which  the  method  applies.  The  remaining  risk  for  this  item  simply  highlights  the 
fact  that  the  deterministic  knockdown  factor  that  is  traditionally  used  to  determine  the 
inspection  intervals  is  not  particularly  useful  for  reduction  of  risk,  as  the  risk  is  not  explicitly 
calculated  at  any  time.  Note  that  the  after  case  is  used  in  the  whole  of  this  document  and 
analysis. 


0  5000  10000  15000 


Flight  Hours 


- IE-5 


- IE-7 


Before 

—  —  —  After 


Figure  26.  Baseline  Risk  of  DTA  188  Before  and  After  Incorporating  Residual  Stress 
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There  are  several  other  locations  in  the  16  location  subset  for  which  the  modified  crack 
growth  analysis  applies.  These  include  DTAs  055,  164,  165,  057B,  and  124B.  The  analysis  to 
incorporate  residual  stresses  for  these  locations  has  yet  to  be  conducted.  The  baseline  risks 
and  associated  excessively  high  TPM  results  should  be  significantly  reduced  once  these 
modified  crack  growth  analyses  are  incorporated. 

6.5.2.  Sensitivity  Analysis 

Before  the  residual  stresses  were  accounted  for  (as  discussed  in  the  previous  section)  a 
general  sensitivity  analysis  was  conducted  on  two  high  risk  DTA  items  (055  and  188)  to 
determine  if  there  was  a  single  input  or  combination  of  inputs  to  RBDMS  that  was 
responsible  for  causing  the  high  risks  due  to  excessive  conservatism.  An  attempt  was  made  to 
utilize  less  conservative  parameters  for  EIFS,  Fracture  Toughness,  the  spectrum,  and  the 
maximum  stress  distribution.  The  severity  of  each  was  reduced  one-at-a-time  and 
combinations  were  tried  as  well. 

This  analysis  did  not  lead  to  any  major  insights  and  the  results  are  therefore  omitted 
from  this  report.  The  risks  of  the  various  modified  scenarios  did  not  identify  any  input 
parameter  as  the  cause  of  the  high  risks. 

6.6.  Next  Steps 

6.6.1.  Full  Set  of  Locations 

The  analysis  of  the  sixteen  location  subset  is  nearing  completion.  The  next  step  is  to 
perform  a  similar  analysis  for  the  full  set  of  5 1  Aircraft  C/D  wing  locations. 

6.6.2.  NDE  False  Call  Rate 

Currently  the  false  call  rate  associated  with  NDE  inspections  is  assumed  to  be  zero. 
This  is  not  realistic.  The  team  is  investigating  what  a  correct  value  for  use  in  the  cost  model 
may  be  for  the  various  NDE  techniques  employed  in  this  study. 

6.6.3.  Remaining  High  Risk  Locations 

Investigation  of  the  high  risk  locations  that  are  not  affected  by  the  incorporation  of 
residual  stresses  is  ongoing  and  is  being  conducted  on  a  case-by-case  basis. 

6.6.4.  System  Level  Risk  Assessment 

Thus  far  in  the  analysis  only  the  risk  of  individual  components  has  been  considered. 
This  is  an  essential  first  step  to  quantifying  the  risk  of  the  structural  system.  Additional 
research  needs  to  be  undertaken  to  determine  the  most  accurate  way  to  calculate  the  SFPOF 
of  the  wing  system  as  a  whole  in  order  to  most  appropriately  address  the  requirements  of 
MIL-STD- 1 530C. 
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6.6.5.  Sensor  Degradation 

Thus  far  the  capability  of  the  in-situ  sensors  used  in  the  analysis  has  been  considered 
constant  throughout  the  lifetimes  of  these  sensors.  That  is,  it  has  been  assumed  that  the 
sensors  will  not  degrade.  However,  in  the  present  cost  analysis,  the  sensors  for  each  location 
are  assumed  to  be  replaced  five  times  within  the  lifespan  of  the  design.  In  other  words,  all  the 
sensors  will  be  replaced  on  average  every  12  years  based  on  the  5  replacements  in  60  years 
lifespan  assumption.  This  suggests  that  some  degradation  of  capability  is  expected  to  occur 
within  these  12  year  periods. 

Research  for  the  degradation  model  will  be  focused  on  two  areas.  The  first  is  to  study 
the  sensor  reliability  model  and  determine  when  the  most  appropriate  time  to  replace  these 
sensors  is.  The  second  is  to  study  the  sensor  degradation  model  and  to  determine  its  impact 
on  the  POD  capability,  if  any. 

6.6.6.  Sensor  Dependency 

Thus  far  successive  in-situ  sensor  readings  used  in  the  CBM+SI  analysis  have  been 
considered  independent.  However,  unlike  the  traditional  infrequent  NDI  type  of  inspections, 
recurring  inspections  for  an  automated  in-situ  sensor  system  should  be  modeled  as  dependent 
due  to  the  largely  repeatable  aspects  of  the  sensor  and  data  collection  system.  This 
dependency  could  impact  the  POD  curve.  Research  for  the  dependency  issue  will  be  focused 
on  the  following  areas: 

•  How  to  design  the  in-situ  sensor  system  so  that  successive  measurements  are  as 
independent  as  possible 

•  How  to  determine  the  coefficient  of  correlation  given  various  environmental 
conditions 

•  How  to  adjust  the  POD  curve  resulting  from  the  correlation 

•  How  to  compute  the  percentage  of  cracks  detected  given  the  correlation 
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7.  Cost  Analysis  Progress 

7.1.  Driver  Analysis 

As  part  of  any  Cost  Benefit  Analysis  a  Driver  Analysis  must  first  be  performed  to 
determine  what  components  would  most  benefit  from  health  monitoring.  Last  March 
Aircraft  Availability  and  Maintenance  Hours  were  used  as  criteria  for  determining  what 
components  would  be  analyzed  as  part  of  the  Cost  Benefit  Analysis. 

7.2.  Cost  Benefit  Analysis  Tool 

The  Cost  Benefit  Analysis  (CBA)  Tool  is  an  Excel  spreadsheet  designed  to  capture  the 
operational  costs  due  to  Inspections,  Repairs,  and  Failures  of  the  components  in 
consideration  of  a  baseline  system.  The  operational  costs  for  alternative  systems  that  include 
health  monitoring  are  also  captured  along  with  the  development  and  deployment  of  the 
technology.  The  costs  of  the  alternative  systems  are  each  compared  to  the  baseline  system  to 
determine  the  benefit  of  the  health  management  system  in  terms  of  cost  saving,  in  net  present 
value,  and  also  aircraft  availability. 

The  present  CBA  tool  is  carried  over  from  the  Phase  I  work.  The  Phase  I  tool  was 
extended  to  support  additional  Control  Points  along  with  new  concepts  of  operations  such  as 
the  inclusion  of  false  calls  in  the  cost  assessments  and  the  ability  to  optionally  have  two 
inspection  types  at  a  single  location.  With  the  current  subset  the  aircraft  is  not  permanently 
grounded  due  to  failure,  however,  this  capability  has  been  kept  in  the  tool  for  future  use. 

The  CBA  tool  has  5  primary  tabs  which  are  used  to  calculate  the  costs  of  a  single 
configuration.  Note  that  there  are  several  secondary  tabs  which  support  these  tabs,  and  they 
will  be  discussed  in  the  next  section  of  this  report.  The  primary  tabs  are  as  follows: 

•  Main 

o  Summarizes  the  results  of  the  workbook,  including  TPMs 

•  Baseline 

o  Captures  the  operational  costs  of  the  baseline  strategy  for  the  lifecycle 

•  Modified 

o  Captures  the  operational  costs  of  the  modified  strategy  for  the  lifecycle 

•  SHM  Costs 

o  Spreads  the  costs  captured  in  the  Baseline,  Modified,  and  CostBreakdown  tabs 
and  determines  their  net  present  values  for  the  lifecycle  of  interest 

•  CostBreakdown 

o  Captures  the  costs  associated  with  the  development  of  the  health  monitoring 
system  from  requirements  development  through  operations  and  sustainment 

7.2.1.  CostBreakdown  Tab 

The  CostBreakdown  tab  buckets  costs  into  several  categories:  System  Design  and 
Development,  Initial  Production,  Production  and  Deployment,  and  finally  Operating  and 
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Sustainment.  The  costs  are  determined  using  engineering  estimates  or  by  similarity  to 
existing  systems. 

7.2. 1 . 1 . 1 .  System  Design  and  Development 

The  System  Design  and  Development  costs  for  the  health  management  system  include: 
requirements  development,  system  design,  supplier  support,  prototype  hardware,  and 
qualification  testing.  These  costs  are  for  the  on-board  system  and  the  ground  system  that 
collects  and  analyzes  the  data.  The  on-board  costs  are  dependent  on  the  number  of  Control 
Points  and  their  similarity  to  one  another.  A  ’’Dependency  Factor”  between  zero  and  one  is 
used  as  a  proxy  for  similarity,  which  is  closer  to  one  if  the  Control  Points  are  very  similar 
and  closer  to  zero  if  they  are  very  dissimilar.  For  a  group  of  similar  Control  Points,  the  cost 
of  determining  requirements,  design,  and  qualification  of  the  health  monitoring  system  can 
be  shared.  Our  subject  matter  expert  suggests  that,  for  the  16  location  subset  utilized  in  this 
document,  a  Dependency  Factor  of  0.5  is  appropriate. 

7.2.1. 1.2.  Initial  Production 

The  Initial  Production  costs  are  for  the  procurement  of  flight  qualified  hardware, 
installation  of  the  health  management  system  by  Boeing  personnel,  initial  training,  and 
evaluation  of  the  health  management  system  prior  to  going  into  production. 

7.2.1. 1.3.  Production  and  Deployment 

The  Production  and  Deployment  costs  are  for  the  procurement  of  the  complete  sensor 
suite  of  all  aircraft  in  the  analysis  and  their  ground  stations,  the  installation  of  health 
management  system  both  on-board  and  off-board,  and  some  additional  training  of  personnel. 
It  is  assumed  the  procurement  and  installation  of  all  hardware  and  software  occurs  in  the  first 
few  years  of  the  program. 

7.2.1. 1.4.  Operational  and  Sustainment 

The  Operational  and  Sustainment  costs  are  for  the  collecting  and  storage  of  the  health 
management  data,  replacement  of  sensors  and  ground  stations,  and  expected  software 
updates  over  the  life  cycle  of  the  system. 

7.2.2.  Baseline  Tab 

The  Baseline  tab  sums  the  operational  costs  for  all  Control  Points  of  interest  in  terms  of 
inspections  and  repairs.  In  the  Baseline  scenario  there  is  only  one  inspection  type:  NDE.  The 
tool  requires  several  inputs  from  RBDMS  and  several  from  our  local  SMEs. 

From  RBDMS  we  receive  the  inspection  intervals,  SFPOF  Before  and  After 
inspections,  and  PCD  for  Small,  Medium,  and  Large  cracks.  SMEs  provide  information  such 
as:  Labor  Hours/Inspection  of  the  Control  Point,  additional  costs  and  materials  that  may  be 
required,  NDE  False  Call  rate,  the  number  of  similar  potential  failure  locations  in  the  Control 
Point,  aircraft  downtime  due  to  inspections,  and  part  replacement  for  when  the  damage  is 
considered  too  great  to  be  repaired.  From  this  information  the  cost  and  aircraft  downtime  are 
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calculated  for  each  Control  Point.  As  stated  in  previous  sections  the  NDE  False  Call  rate  is 
currently  set  to  zero  and  does  not  factor  into  the  costs  for  this  analysis. 

From  these  inputs  the  tool  calculates  the  number  of  flights  between  inspections,  the 
probability  of  failure  for  each  flight  interval  for  the  Control  Points  of  interest,  the  probability 
of  detection  at  each  inspection  of  cracks  of  small,  medium,  and  large  sizes,  the  expected  cost 
of  repairs  for  all  cracks,  the  number  of  inspections  for  the  life  of  the  aircraft,  the  costs  for  all 
the  inspections,  and  the  downtime  of  the  aircraft  for  these  inspections.  All  costs  are  also 
converted  to  present  day  dollar  values  for  calculation  of  the  Net  Present  Value. 

Note  that  to  calculate  the  probability  of  failure  for  each  flight  in  an  interval  the  tool 
assumes  that  the  SFPOF  increases  linearly  between  inspections. 

7.2.3.  Modified  Tab 

The  Modified  tab  is  very  similar  to  the  Baseline  tab.  The  most  notable  difference  is  that 
in  the  Baseline  tab  there  is  only  one  inspection  type:  NDE.  In  the  Modified  tab  there  are  three 
inspection  types:  in-situ  without  accessibility  penalties,  in-situ  with  accessibility  penalties, 
and  NDE.  Recall,  for  inaccessible  locations  there  is  a  500  man  hour  penalty  for  repairs  of 
inaccessible  locations,  as  well  as  an  8  hour  downtime  penalty.  This  is  implemented  by  always 
using  the  first  inspection  type  for  field-accessible  locations  and  for  inaccessible  locations  at 
the  depot,  and  always  using  the  second  inspection  type  for  inaccessible  locations  away  from 
the  depot. 


7.2.4.  SHM  Costs  Tab 

The  costs  captured  on  the  CostBreakdown  tab,  Baseline  tab,  and  Modified  tab  are 
spread  across  the  years  of  implementation  on  the  SHM  Cost  tab.  Net  Present  Values  are  then 
calculated  for  the  life  of  the  program  for  both  the  Baseline  and  Modified  designs. 

7.2.5.  Main  Tab 

The  Main  tab  is  use  to  collect  the  costs  and  downtimes  calculated  in  the  previous  four 
tabs.  The  costs  and  downtimes  are  then  summed  and  compared  between  the  Baseline  and 
Modified  Designs.  The  costs  are  compared  dollar  to  dollar  but  the  downtime  is  used  to 
determine  the  aircraft  availability.  The  Baseline  availability  is  pulled  from  reliability  metrics 
from  the  service  and  the  Modified  availability  is  calculated  from  Modified  systems  downtime 
compared  to  the  Baseline  systems  downtime. 

7.3.  Results  Import  Macro 

The  CBA  tool  described  in  the  previous  section  calculates  the  TPMs  for  a  single 
configuration.  As  stated  previously,  in  the  analysis  of  the  16  location  subset  there  are  5184 
in-situ  configurations  to  consider.  The  task  of  running  the  many  sets  of  results  through  the 
cost  model  is  accomplished  with  a  VBA  (Visual  Basic  for  Applications)  macro. 
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Within  the  workbook  which  houses  the  CBA  tool,  three  additional  tabs  were  added  to 
support  the  macro  and  programmatically  modify  the  contents  of  the  primary  CBA  tabs:  the 
MACRO,  DTA  Info,  and  Configs  tabs. 


7.3.1.  MACRO  Tab 

There  are  two  tables  in  the  MACRO  tab.  The  first  lists  the  options  available  for  each 
DTA,  along  with  the  folder  containing  the  RBDMS  results,  the  results  filename  conventions, 
and  the  false  call  rates  associated  with  each  available  option.  See  Figure  27.  At  this  time  the 
Baseline  and  Risk-based  options  have  no  false  call  rate  associated  with  them.  Also,  note  that 
only  the  Mix  options  have  a  different  Pr(FA)  for  the  two  inspection  types.  For  the  other 
options,  the  inspection  is  the  same  in  Type  1  and  Type  2.  However,  Type  2  includes  the 
accessibility  penalties  and  Type  1  does  not.  Note  that  throughout  the  CBA  the  cells  with  a 
yellow  background  are  user-modifiable,  and  the  rest  are  either  fixed  or  calculated  cells. 


Results 

Folder  Path 

C:\ResultsFolder 

Run  Names 

Filename  Suffix 

SHM  Type  1 
Pr(FA) 

SHM  Type  2 
Pr(FA) 

Base 

base.out 

Risk 

risk.out 

HF3 

HF3.out 

0.01 

0.01 

HF6 

HF6.out 

0.01 

0.01 

MF3 

MF3.out 

0.001 

0.001 

MF6 

MF6.out 

0.001 

0.001 

LF3 

LF3.out 

0.00001 

0.00001 

LF6 

LF6.out 

0.00001 

0.00001 

MixHL3 

MixHL3.out 

0.01 

0.00001 

MixHL6 

MixHL6.out 

0.01 

0.00001 

MixML3 

MixML3.out 

0.001 

0.00001 

MixML6 

_MixML6.out 

0.001 

0.00001 

Figure  27.  Available  Inspection  Options 

The  second  table  in  the  MACRO  tab  specifies  the  options  selected  for  the  16  DTAs  in 
the  “Current  Run”.  The  Current  Run  refers  to  the  configuration  that  is  currently  being  used  in 
the  remainder  of  the  cost  model.  See  Figure  28.  The  option  selected  for  each  DTA  specifies 
both  what  the  RBDMS  results  filename  is  called  and  whether  this  option  calls  for  SHM  or 
NDE.  The  number  of  SHM  locations  is  counted  in  the  cell  at  the  bottom  right  of  the  figure, 
and  this  drives  the  count  of  SHM  locations  in  the  tab  CostBreakdown  (described  in  the 
previous  section). 
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Current  Run  DTA 

Filename 

NDE  orSHIN/f 

LF3 

dta055 

dta055_LF3.out 

SHM 

LF6 

dta057B 

dta057B_LF6.out 

SHM 

Risk 

dta097 

dta097_risk.out 

NDE 

MixHL3 

dtall5 

dtall5_MixHL3.out 

SHM 

LF6 

dtal24B 

dtal24B_LF6.out 

SHM 

MF3 

dtal34B 

dtal34B_MF3.out 

SHM 

MixHL3 

dtal38B 

dtal38B_MixHL3.out 

SHM 

LF6 

dtal43 

dtal43_LF6.out 

SHM 

MF3 

dtal44 

dtal44_MF3.out 

SHM 

Risk 

dtal64 

dtal64_risk.out 

NDE 

LF3 

dtal65 

dtal65_LF3.out 

SHM 

Risk 

dtal66B 

dtal66B_risk.out 

NDE 

Risk 

dtal84 

dtal84_risk.out 

NDE 

Risk 

dtal87 

dtal87_risk.out 

NDE 

Risk 

dtal88 

dtal88_risk.out 

NDE 

Risk 

dta203 

dta203_risk.out 

NDE 

#  of  SHM  in  Current  Run 

9 

Figure  28.  Current  Configuration  for  CBA 

In  addition  to  these  two  tables,  this  sheet  contains  the  VBA  macro  itself  behind-the- 
scenes.  Also,  the  button  which  instructs  the  macro  to  run  is  located  below  these  tables  in  the 
MACRO  tab. 

7.3.2.  DTA  Info  Tab 

The  information  specific  to  each  control  point  is  located  in  this  tab  as  a  matrix.  The 
values  here  propagate  through  the  Baseline  and  Modified  tabs  of  the  CBA  tool.  A  screenshot 
is  omitted  as  this  information  was  presented  in  Section  4  of  this  report. 


7.3.3.  Configs  Tab 

The  list  of  configurations  to  be  run  through  the  macro  is  contained  in  this  tab.  In  this 
tab  there  is  a  column  corresponding  to  each  DTA  item  and  5184  rows  in  which  the  inspection 
options  are  specified  for  each  DTA.  Additional  columns  are  located  to  the  right  of  this  table 
for  the  TPM  results  corresponding  to  each  configuration.  The  first  several  rows  are  shown 
below  in  two  pieces  in  Figure  29.  Only  the  first  TPM,  Net  Present  Value,  is  shown  due  to 
space  restrictions  in  this  document.  Note  that  each  row  of  this  table  is  unique,  representing 
one  of  the  many  configurations  under  consideration. 


57 

Approved  for  public  release;  distribution  unlimited. 


dta055 

dta057l 

dta097 

dtall5 

dtal24l 

dtal34 

dtal38l 

dtal43 

dtal44 

Risk 

Risk 

Risk 

MixHL3 

LF6 

MF3 

MixHL3 

LF6 

MF3 

LF3 

Risk 

Risk 

MixHL3 

LF6 

MF3 

MixHL3 

LF6 

MF3 

Risk 

LF6 

Risk 

MixHL3 

LF6 

MF3 

MixHL3 

LF6 

MF3 

LF3 

LF6 

Risk 

MixHL3 

LF6 

MF3 

MixHL3 

LF6 

MF3 

dtal64 

dtal65 

dtal66l 

dtal84 

dtal87 

dtal88 

dta203 

NPV 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

$1,787,958,197.73 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

$1,812,138,981.07 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

$1,790,023,406.36 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

Risk 

$1,814,205,781.37 

Figure  29.  First  Several  Rows  of  Configs  Tab 

The  macro  starts  with  the  first  row  of  this  table,  copying  and  pasting  the  contents  into 
the  first  column  of  the  table  of  the  MACRO  tab  shown  in  Figure  28.  The  results  file  names 
indicated  in  that  table  are  then  sequentially  imported  by  the  macro.  The  parameters 
appropriate  to  each  option  propagate  through  the  workbook  as  well,  ultimately  calculating 
the  TPMs.  The  value  of  each  TPM  associated  with  that  configuration  are  then  copied  and 
pasted  into  the  results  columns  of  the  Configs  tab,  such  as  NPV  in  Figure  29. 

Once  this  has  been  completed  for  the  first  row,  the  macro  moves  to  the  second  row  and 
repeats  the  process.  This  continues  until  all  5184  configurations  have  had  their  results 
imported  and  their  TPMs  tabulated. 

The  macro  takes  approximately  22  minutes  to  run  for  5184  configurations. 

7.4.  Financial  Uncertainty  Analysis 
7.4.1.  Summary  of  March  Report 

The  previous  progress  report  introduced  the  strategy  and  identified  the  inputs  to  the  cost 
model  that  are  to  be  considered  random  variables  in  the  financial  uncertainty  analysis.  Also, 
the  report  contained  some  preliminary  results  of  this  analysis  concerning  the  Phase  I  business 
case. 


The  ultimate  goal  is  to  conduct  a  financial  uncertainty  analysis  for  Phase  II.  Because 
the  cost  model  for  Phase  II  is  incomplete,  it  was  determined  that  the  method  should  be 
demonstrated  for  the  completed  cost  model  of  Phase  I.  As  the  financial  uncertainty  analysis 
for  Phase  I  has  been  completed,  it  is  written  up  here  as  a  stand-alone  section.  To  follow  the 
discussion  one  need  not  refer  to  the  previous  progress  report. 
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7.4.2.  Motivation 


Phase  I  included  a  business  case  analysis  for  each  of  the  following  inspection  strategies: 

•  Baseline 

o  Current  inspection  methodology 

•  ndeRisk 

o  Risk-based  inspections  (no  SHM) 

•  NDESHM 

o  Combined  NDE/SHM  strategy 

•  SHM300 

o  SHM  only  w/  300  FH  intervals 

•  SHM400 

o  SHM  only  w/  400  FH  intervals 

•  SHM500 

O  SHM  only  w/  500  FH  intervals 


The  costs  of  the  baseline  strategy  are  known  and  need  not  be  analyzed  here.  The 
ndeRisk  strategy  contains  far  less  uncertainty  due  to  the  fact  that  there  is  no  SHM  system  in 
use  in  that  case.  In  addition  to  these  strategies,  there  were  two  failure  options  in  Phase  I,  each 
offering  two  choices: 

•  Number  of  locations  where  failure  can  occur 

o  Multiple  similar  locations 
o  Single  location 

•  Consequence  of  failure 

o  Foss  of  platform 
o  Replacement  of  bulkhead 


Of  these  failure  options,  the  most  realistic  and  conservative  case  is  thought  to  be  that 
which  considers  failure  at  multiple  locations  and  loss  of  the  platform  in  the  event  of  failure. 
The  financial  uncertainty  analysis  is  conducted  utilizing  the  versions  of  the  cost  model  that 
are  associated  with  these  options. 

The  original  deterministic  cost  benefit  analysis  was  conducted  for  each  of  the  strategies 
using  an  Excel  spreadsheet  model.  The  criteria  for  comparison  of  the  strategies  were  three 
Technical  Performance  Measures  (TPMs):  %  Fleet  Non-Mission  Capable  (%NMC),  Fife 
Cycle  Cost  (FCC),  and  Net  Present  Value  (NPV).  It  was  noted  at  the  time  of  the  Phase  I 
final  briefing  that  several  of  the  inputs  to  the  business  case  analysis  are  uncertain  estimates 
for  what  will  occur  in  the  future. 

To  understand  how  uncertainty  regarding  the  estimates  of  these  input  quantities  may 
affect  the  results  of  the  TPMs  an  analysis  has  been  undertaken  to  determine  the  variability  of 
the  TPMs  with  regard  to  the  uncertainty  of  the  input  variables. 
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7.4.3.  Strategy 


To  conduct  the  uncertainty  analysis,  some  means  of  placing  probability  distributions  on 
the  input  variables  was  required.  It  was  determined  that  the  most  straightforward  method  for 
accomplishing  this  was  to  use  the  Phoenix  Integration  software  ModelCenter. 

ModelCenter  is  capable  of  creating  models  which  allow  data  to  flow  from  one  software 
application  to  another.  In  addition,  Monte  Carlo  sampling  can  be  performed  in  which  a 
probability  distribution  may  be  assigned  to  one  or  more  input  variables.  ModelCenter  refers 
to  this  as  a  probabilistic  analysis.  Here,  ModelCenter  is  used  with  a  single  application, 
Microsoft  Excel.  The  input  variables  of  the  Excel-based  cost  model  are  randomized  using 
ModelCenter  and  a  Monte  Carlo  simulation  is  performed. 

The  essential  strategy  for  performing  the  financial  uncertainty  analysis  of  the  Phase  I 
business  case  analysis  is  as  follows: 

•  Identify  the  input  variables  that  are  uncertain 

•  Acquire  expert  opinion  to  define  probability  distributions  for  each  input  variable 

•  Perform  a  Monte  Carlo  simulation  for  each  inspection  strategy  using  ModelCenter 

•  Characterize  the  distribution  of  each  of  the  three  TPMs  in  each  model 

•  Perform  a  sensitivity  analysis  to  identify  the  critical  model  inputs 

•  Assess  what  (if  any)  implications  this  has  for  the  business  case  analysis  of  Phase  I 

7.4.4.  Uncertain  Cost  Model  Input  Variables 

The  team’s  SHM  subject  matter  expert  (SME)  identified  those  cost  model  input 
variables  which  are  uncertain  estimates,  and  also  identified  the  best-  and  worst-case  values 
for  these  variables.  The  uncertainty  analysis  for  each  of  the  SHM  strategies  is  conducted 
twice,  utilizing  two  distributional  assumptions  for  these  variables.  In  the  first  case  (uniform 
case),  each  cost  is  represented  with  a  uniform  distribution  between  the  low  and  high  values. 
In  the  second  case  (normal  case),  each  cost  is  represented  with  a  normal  distribution  with  the 
mean  placed  evenly  between  the  best-  and  worst-case  values  and  the  standard  deviation 
chosen  such  that  the  low  and  high  values  are  each  two  standard  deviations  from  the  mean. 
The  SME  also  identified  two  variables  that  are  excellent  point  estimates,  yet  still  are  not 
certain.  These  variables  are  assigned  a  normal  distribution  with  10%  CoV  in  both  the 
uniform  and  normal  cases. 

Lastly,  the  SME  indicated  that  two  variables  of  the  original  cost  model  are  very  highly 
correlated.  These  are  Complete  Requirements  Derivation  and  Design  and  Associated 
Engineering  Costs  of  sheet  “CostBreakdown”  in  the  cost  model.  These  are  assumed  in  this 
analysis  to  be  perfectly  linearly  correlated.  Hence  the  former  random  variable  is  randomized 
in  ModelCenter,  and  the  latter  is  determined  within  Excel  utilizing  a  linear  relationship  in 
proportion  to  the  original  cost  model  estimates. 
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The  random  variables  for  the  SHM  cases  are  described  below.  The  first  part  of  the 
variable  name  indicates  the  sheet  of  the  cost  model  where  it  is  location.  Cost-  corresponds  to 
sheet  CostBreakdown,  Main-  to  sheet  Main,  and  Mod-  to  sheet  Modified. 

•  CostAircraftlntegration 

o  Hours  required  to  determine  how  to  integrate  the  sensors  in  the  aircraft 

•  CostCompleteSensorSet 

o  Hardware  cost  of  the  sensors  themselves 

•  CostDerivationAndDesign 

o  Hours  required  to  design  and  develop  the  sensor  system  capability 

•  CostDesignValidation 

o  Hours  for  prototyping  and  testing 

•  CostlnitProdTeamMan 

o  Hours  of  labor  in  the  initial  production  period  to  gain  confidence  in  the  data 
being  collected  by  the  system 

•  CostMatureGroundSystem 

o  Cost  of  off-the-shelf  data  acquisition  system 

•  CostQualAndCertSupport 

o  Hours  preparing  certification  paperwork 

•  CostRepairReplaceSensors 

o  Average  number  of  times  the  sensor  system  will  be  replaced  over  the  life 

•  MainlnstallLaborHours 

o  Hours  to  install  the  sensors  themselves 

•  ModLaborHrsPerlnspec 

o  Hours  required  to  perform  an  inspection 

•  ModLaborHrsPerRepair 

o  Hours  required  to  perform  a  repair 

•  CostMaterialsCost 

o  Cost  of  prototype  unit  prototyping  and  testing 

•  CostSupplierEngEffort 

o  Cost  for  design  and  building  of  prototype 

The  distributions  assigned  to  each  of  the  random  variables  for  the  SHM  strategies  of  the 
financial  uncertainty  analysis  are  shown  in  Table  15  below,  along  with  the  distributions 
assigned  for  each  of  the  two  analysis  cases.  Note  that  the  best-  and  worst-case  values  are 
often  higher  than  those  used  in  the  original  estimate.  For  this  analysis  it  was  decided  that  the 
estimated  distributions  should  error  on  the  side  of  conservatism.  This  will  be  evident  when 
the  results  are  presented  in  a  later  section  as  the  distributions  of  the  TPMs  will  be  more  or 
less  preferable  than  were  the  original  point  estimates. 
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ModelCenter  Variable  Original  Best  Worst  Normal  Normal 


Name 

Units 

Estimate 

Case 

Case 

Mean 

St  Dev 

CostAi  rcraftl  ntegration 

hrs 

10 

8 

16 

12 

2 

CostCompleteSensorSet 

$ 

750 

500 

750 

625 

62.5 

CostDerivationAndDesign 

hrs 

160 

160 

960 

560 

200 

CostDesignValidation 

hrs 

80 

160 

480 

320 

80 

CostlnitProdTeamMan 

hrs 

960 

640 

1280 

960 

160 

CostMatureGroundSystem 

$ 

5000 

3000 

10000 

6500 

1750 

CostQualAndCertSupport 

hrs 

80 

80 

160 

120 

20 

CostRepairReplaceSensors 

# 

2 

2 

5 

3.5 

0.75 

MainlnstallLaborHours 

hrs 

5 

5 

10 

7.5 

1.25 

ModLaborHrsPerlnspec 

hrs 

0.5 

0.5 

1 

0.75 

0.125 

ModLaborHrsPerRepair 

hrs 

2 

1.5 

2.5 

2 

0.25 

CostMaterialsCost 

$ 

2000 

2000 

200 

CostSupplierEngEffort 

$ 

5000 

5000 

500 

Table  15.  Variables  and  Associated  Distributions  for  SHM  Strategies 


In  addition  to  the  SHM  strategies  is  the  ndeRisk  strategy.  The  uncertainty  analysis  for 
this  contains  fewer  random  variables,  but  is  similarly  conducted  utilizing  both  uniform  and 
normal  distributional  assumptions.  The  random  variables  for  the  ndeRisk  strategy  and  the 
assigned  distributions  are  shown  in  Table  16. 


ModelCenter  Variable 

Name 

Units 

Original 

Estimate 

Best 

Case 

Worst 

Case 

Normal 

Mean 

Normal 

St  Dev 

CostDerivationAndDesign 

hrs 

1920 

1920 

2880 

2400 

240 

CostlnitProdTeamMan 

hrs 

640 

640 

1280 

960 

160 

ModLaborHrsPerlnspec 

hrs 

3 

3 

4 

3.5 

0.25 

ModLaborHrsPerRepair 

hrs 

1.5 

1 

2 

1.5 

0.25 

Table  16.  Variables  and  Associated  Distributions  for  ndeRisk  Strategy 
7.4.5.  TPM  Results 

The  most  important  TPM  is  the  net  present  value  (NPV).  The  life  cycle  cost  (LCC)  is 
very  similar  to  the  NPV.  The  NPV  considers  the  time  value  of  money,  but  the  LCC  does  not. 
Note  that  both  NPV  and  LCC  represent  costs,  that  is,  lower  is  better.  The  third  TPM,  percent 
non-mission-capable  (%NMC),  was  shown  in  Phase  I  to  be  negligibly  affected  by  the  various 
strategies.  Due  to  these  considerations,  the  majority  of  discussion  in  this  section  refers  to  the 
NPV.  For  reference,  Table  17  contains  the  totality  of  results  for  each  of  the  TPMs. 
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TPM 

Strategy 

Uniform 

Mean 

Uniform 

St  Dev 

Normal 

Mean 

Normal 

St  Dev 

NPV 

ndeRisk 

24.692 

0.120 

24.692 

0.102 

($m) 

NDESHM 

15.371 

0.807 

15.377 

0.698 

SHM300 

5.703 

0.839 

5.698 

0.702 

SHM400 

5.698 

0.806 

5.713 

0.689 

SHM500 

9.115 

0.805 

9.158 

0.678 

LCC 

ndeRisk 

35.901 

0.150 

35.900 

0.126 

($m) 

NDESHM 

22.090 

1.181 

22.100 

1.021 

SHM300 

7.887 

1.227 

7.878 

1.028 

SHM400 

7.879 

1.179 

7.903 

1.009 

SHM500 

12.900 

1.178 

12.963 

0.993 

PercNMC  ndeRisk 

0.0536% 

0.0042% 

0.0535% 

0.0036% 

NDESHM 

0.0187% 

0.0031% 

0.0188% 

0.0027% 

SHM300 

0.0384% 

0.0066% 

0.0383% 

0.0057% 

SHM400 

0.0309% 

0.0052% 

0.0305% 

0.0045% 

SHM500 

0.0236% 

0.0040% 

0.0234% 

0.0035% 

Table  17:  All  Results  for  Each  Strategy 


In  Table  17  above  you  can  see  that  the  resulting  standard  deviations  for  each  TPM  are 
slightly  lower  for  the  normal  case.  This  is  due  to  the  fact  that  the  normal  distribution  has 
more  weight  at  the  mean  value  than  does  the  uniform  distribution. 

Figure  30  and  Figure  31  below  depict  dotplots  of  NPV  for  each  of  the  strategies  for  the 
uniform  case  and  normal  case,  respectively.  Each  point  in  the  dotplot  is  a  single  realization  of 
the  Monte  Carlo  analysis.  The  values  have  been  jittered  vertically  so  that  the  density  of  the 
points  can  be  easily  visualized.  The  original  point  estimates  of  the  Phase  I  business  case 
analysis  are  shown  as  red  on  yellow  diamonds  in  the  figure.  Note  that  in  all  SHM  strategies 
the  NPV  is  vastly  preferable  to  that  of  the  ndeRisk  strategy.  The  variability  of  the  NPV  is  not 
such  that  the  original  discoveries  of  the  deterministic  cost  benefit  analysis  of  Phase  I  need  not 
be  questioned. 


63 

Approved  for  public  release;  distribution  unlimited. 


ndeRisk  - 


NDESHM  - 
SHM300  - 
SHM400  - 
SHM500  - 

5  10  15  20  25 

NPV  (cost  in  millions  of  $) 

Figure  30.  Dotplot  of  NPV ;  Uniform  Case 
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Figure  31.  Dotplot  of  NPV;  Normal  Case 
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7.4.6.  Sensitivity  Analysis  for  NPV 

To  better  understand  the  variables  which  drive  the  NPV,  a  sensitivity  analysis  was 
performed.  The  sensitivity  analysis  is  not  performed  for  LCC  as  LCC  and  NPV  both  measure 
costs  and  thus  the  sensitivity  analyses  will  yield  similar  results.  Also,  this  analysis  was  not 
conducted  for  %NMC  since  it  is  intuitively  obvious  which  variables  drive  the  downtime: 
ModLaborHrsPerlnspec,  ModLaborHrsPerRepair,  and  MainlnstallLaborHours.  Finally,  this 
analysis  was  not  performed  for  the  ndeRisk  case  as  the  engineering  manpower  variables 
associated  with  that  are  considered  to  be  well  understood  (in  contrast  to  SHM  development 
and  maintenance  costs). 
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This  analysis  consists  first  of  a  correlation  analysis  which  determines  which  input 
variables  are  most  highly  linearly  correlated  with  the  NPV.  Then,  a  multiple  linear  regression 
analysis  is  conducted  to  find  the  amounts  by  which  the  NPV  will  change  for  given  changes  in 
the  inputs.  Note  that  each  of  these  assumes  a  linear  relationship  between  the  inputs  and  NPV. 
This  linearity  assumption  can  be  validated  by  viewing  a  scatter  plot  of  each  input  variable 
with  NPV.  All  but  one  of  these  plots  are  noise  for  the  most  part  as  the  individual  correlations 
are  not  strong.  A  typical  example  is  shown  below  in  Figure  32,  along  with  the  plot  of  the  one 
variable  which  exhibits  nearly  perfect  linear  correlation.  This  variable  is  discussed  below. 


o 


200  600  2.0  3.0  4.0  5.0 

CostDerivationAndDesign  CostRepairReplaceSensors 

Figure  32.  Scatter  Plots  of  NPV  Versus  Two  Input  Variables 


The  results  of  the  correlation  analysis  are  shown  below  in  Table  18.  The  correlations 
shown  are  known  as  Pearson’s  Correlation  Coefficients,  which  measure  the  degree  of 
linearity  in  a  relationship.  This  value  has  a  range  between  -1  and  +1,  where  -1  indicates  a 
perfect  linear  relationship  with  a  negative  slope,  +1  indicates  a  perfect  linear  relationship 
with  a  positive  slope,  and  0  indicates  no  linear  relationship. 


NDESHM 

SHM300 

SHM400 

SHM500 

CostRepairReplaceSensors 

0.99 

0.98 

0.98 

0.99 

ModLaborHrsPerlnspec 

0.08 

0.18 

0.10 

0.12 

Color  Scale 

CostDerivationAndDesign 

0.08 

0.14 

0.07 

0.07 

0.99 

CostlnitProdTeamMan 

0.09 

0.04 

0.07 

0.07 

0.84 

CostCompleteSensorSet 

0.03 

0.06 

0.08 

0.06 

0.70 

Cost  M  at  u  reG  rou  nd  System 

0.02 

0.07 

0.06 

0.09 

0.55 

MainlnstallLaborHours 

0.03 

0.02 

0.04 

0.05 

0.40 

CostDesignValidation 

0.07 

0.04 

0.01 

0.01 

0.26 

CostQualAndCertSupport 

0.04 

-0.01 

0.04 

0.03 

0.11 

CostMaterialsCost 

0.02 

0.04 

0.03 

-0.02 

-0.04 

CostSupplierEngEffort 

0.03 

0.05 

0.01 

-0.02 

CostAircraftlntegration 

0.03 

0.02 

0.02 

-0.01 

ModLaborHrsPerRepair 

-0.02 

0.02 

0.05 

-0.04 

Table  18.  Correlation  Analysis  of  Input  Variables 
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The  variable  CostRepairReplaceSensors  is  responsible  for  the  majority  of  the 
variability  of  NPV.  This  is  not  surprising  as  the  SME  had  previously  indicated  that  the  large 
range  given  between  the  best-  and  worst-case  values  is  necessary  for  this  variable  because  in- 
situ  sensors  of  the  type  used  in  this  analysis  have  simply  not  been  utilized  in  the  field  for 
decades  of  continuous  use.  The  conservative  assumption  was  that  one  would  need  to  replace 
the  entire  system  of  sensors  2  to  5  times  during  the  life  for  each  aircraft,  which  of  course  has 
a  tremendous  effect  on  the  cost  of  maintaining  the  fleet.  The  correct  value  for  this  variable  is 
difficult  to  ascertain  without  performing  a  large  scale,  long  term  experiment. 

The  other  variable  which  has  some  significant  effect  in  the  SHM  strategies  is 
ModLaborHrsPerlnspec.  This  is  also  understandable  as  this  indicates  the  time  required  to 
perform  the  data  download  and  analysis  from  the  SHM  system.  Information  regarding  the 
correct  value  for  this  variable  could  be  obtained  from  a  relatively  inexpensive  experiment. 

To  show  how  the  NPV  changes  with  each  input  variable  a  multiple  linear  regression 
analysis  is  conducted  to  obtain  the  regression  coefficients  for  each  strategy.  The  regression 
coefficients  are  shown  below  in  Table  19.  Note  that  many  of  the  coefficients  are  not  altered 
by  the  strategy.  The  ones  that  change  affect  NPV  through  the  inspection  frequency,  such  as 
ModLaborHrsPerlnspec. 


ModelCenter  Variable 

Name 

Units 

Original 

Estimate 

NDESHM  SHM300 

RegCoefs  RegCoefs 

SHM400  SHM500 

RegCoefs  RegCoefs 

CostAi  rcraftl  ntegration 

hrs 

10 

178 

178 

178 

178 

CostCompleteSensorSet 

$ 

750 

685 

685 

685 

685 

CostDerivationAndDesign 

hrs 

160 

266 

266 

266 

266 

CostDesign  Validation 

hrs 

80 

178 

178 

178 

178 

CostlnitProdTeamMan 

hrs 

960 

339 

339 

339 

339 

CostMatureGroundSystem  $ 

5000 

17 

17 

17 

17 

CostQualAndCertSupport 

hrs 

80 

178 

178 

178 

178 

CostRepairReplaceSensors  # 

2 

918763 

918763 

918763 

918763 

MainlnstallLaborHours 

hrs 

5 

22077 

22077 

22077 

22077 

ModLaborHrsPerlnspec 

hrs 

0.5 

396406 

872093 

673890 

515328 

ModLaborHrsPerRepair 

hrs 

2 

28037 

35424 

38330 

27699 

CostMaterialsCost 

$ 

2000 

0.987 

0.987 

0.987 

0.987 

CostSupplierEngEffort 

$ 

5000 

0.987 

0.987 

0.987 

0.987 

Table  19.  Multiple  Linear  Regression  Coefficients  for  NPV 


The  regression  coefficient  is  the  slope  associated  with  each  input  variable.  That  is,  it  is 
the  amount  by  which  the  NPV  changes  if  the  input  variable  is  increased  by  one  unit.  The 
original  estimate  is  given  to  assist  with  interpretation. 

For  example,  the  coefficient  of  the  first  variable,  CostAircraftlntegration  is  $178  for  all 
strategies.  The  original  estimate  for  this  variable  was  10  hrs.  If  it  turns  out  to  take  11  hours, 
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the  net  present  value  of  costs  for  the  fleet  increases  by  $178  (a  trivial  amount).  For  this 
variable  to  be  relevant  there  would  need  to  be  a  gross  underestimate  here,  on  the  order  of 
hundreds  of  hours. 

Next  consider  CostRepairReplaceSensors.  The  original  estimate  for  the  number  of 
times  the  sensor  system  would  need  to  be  replaced  in  the  life  of  each  platform  was  2.  If 
instead  the  average  number  of  replacements  turns  out  to  be  3  per  aircraft,  the  NPV  of  the 
system  increases  by  $918,763.  This  is  significant.  Recall  the  strong  relationship  between  this 
variable  and  NPV  shown  in  Figure  32  and  Table  18  above.  It  is  crucial  that  this  variable  be 
conservatively  estimated  in  any  cost  benefit  analysis  to  ensure  that  costs  are  not  grossly 
underestimated.  Some  experimentation  may  be  necessary  to  better  estimate  the  durability  of 
this  type  of  in-situ  sensor  system. 

7.4.7.  Conclusion 

This  analysis  has  shown  a  method  for  conducting  a  financial  uncertainty  analysis  for  an 
associated  cost  model.  In  this  particular  example  it  was  shown  that  the  variability  of  the  NPV 
does  not  have  any  significant  implications  regarding  the  conclusions  of  the  Phase  I  business 
case  analysis.  However,  the  sensitivity  analysis  did  identify  that  one  of  the  input  variables  of 
the  cost  model,  CostRepairReplaceSensors ,  was  the  major  driver  of  the  variability  of  NPV. 
Care  should  be  taken  when  estimating  this  parameter  in  future  analyses. 

The  financial  uncertainty  analysis  for  Phase  II  will  commence  when  the  cost  model  for 
that  project  has  reached  a  mature  level.  It  is  the  expectation  of  the  analysis  team  that  the 
method  used  for  Phase  I  can  be  readily  utilized  for  Phase  II. 
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Appendix 


A.l  Control  Point  Information 

Extensive  tables  of  information  regarding  each  of  the  51  control  points  of  the  F- 15  C/D 
wing  systems  are  shown  in  Figure  A-l  and  Figure  A-2. 
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Figure  A-l.  Control  Point  Information  (1  of  2) 
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Figure  A-2.  Control  Point  Information  (2  of  2) 
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A.2  F-15  Methodology  for  Inclusion  of  Residuals  Stresses  in  Crack  Growth  Analysis 

Due  to  Cold  Working  or  Interference  Fit  Fasteners 

The  current  methodology  on  the  F-15  program  to  account  for  residual  stresses  due  to 
interference  fit  fasteners  or  cold  worked  holes  is  conservative.  The  initial  flaw  size  is 
decreased  in  the  crack  growth  analysis  from  a  0.05”  comer  flaw  to  a  0.005”  comer  flaw  and 
no  residual  stresses  are  used  from  cold  working  or  interference  fit  fasteners.  This  is  a 
conservative  approach  compared  to  actually  using  mechanically  induced  residual  stresses  as 
part  of  the  crack  growth  loading  profile.  This  has  been  done  historically  as  it  was  assumed 
that  control  of  cold  working  or  installation  of  interference  fit  fasteners  in  large  areas  (such  as 
the  lower  wing  skin)  may  not  be  maintained. 

This  approach  presents  a  problem  for  the  risk  assessment  tool  PROF.  The  assumed 
initial  flaw  size  in  a  PROF  risk  assessment  is  0.003”  (based  on  an  Equivalent  Initial  Flaw 
Size  approach).  For  normal  fastener  holes,  a  change  from  an  initial  flaw  of  0.05”  to  0.003” 
provides  a  relatively  large  increase  in  life.  But  when  the  flaw  size  only  changes  from  0.005” 
to  0.003”  the  crack  growth  life  has  only  a  small  change.  This  causes  the  risk  assessment  to 
be  exceptionally  severe. 

To  include  residual  stresses  in  crack  growth  analysis,  the  Finite  Element  program 
StressCheck  is  used.  StressCheck  can  handle  non-linear  incremental  plasticity  analysis 
which  can  accurately  determine  residual  stresses  due  to  loading/unloading  that  is  seen  when 
cold  working  a  hole  or  cycling  an  interference  fit  fastener.  For  the  analysis  provided,  all 
holes  are  assessed  as  if  they  are  cold  worked  in  the  StressCheck  program.  Analysis  for 
interference  fasteners  requires  additional  steps  that  cold  working  does  not,  increasing  the 
analysis  time,  which  is  not  felt  justified.  Cold  working  results  are  more  conservative  than 
interference  fit  fastener  results  because  of  the  large  hole  propping  effect  associated  with 
interference  fit  fasteners.  For  this  reason,  only  the  method  of  obtaining  residual  stresses  for 
cold  worked  holes  is  presented  in  this  document. 

The  image  below  shows  a  typical  StressCheck  planar  model  that  would  be  used  to 
extract  a  residual  stress  gradient.  The  plate  material  is  modeled  with  bilinear  material 
properties  and  the  fastener  element  is  given  an  isotropic  steel  property.  A  linear  run  is 
performed  with  an  initial  interference  (the  minimum  interference  as  defined  in  the  Boeing 
Process  Specification  for  a  given  fastener  size).  Once  the  linear  ran  is  complete,  an 
incremental  non-linear  analysis  is  performed  with  two  steps:  mandrel  in  and  mandrel  out. 
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Figure  A-3.  Stress  Check  Detail 


(b)  Bilinear  material 


Figure  A-4.  Bi-linear  Stress-Strain  Diagram 

Bilinear  material  properties  are  chosen  instead  of  other  material  properties  (such  as 
Ramberg-Osgood)  since  bilinear  material  properties  in  planar  analysis  can  accept  a  parameter 
to  control  the  hardening  law. 

Many  engineering  materials  when  cycled  repeatedly  exhibit  kinematic  hardening 
behavior.  StressCheck  however,  requires  all  planar  non-linear  materials  to  use  an  isotropic 
hardening  law  except  for  the  bilinear  material  property  where  it  can  be  specified. 
Consequently,  bilinear  material  properties  are  used. 

Once  the  analysis  is  complete,  a  stress  gradient  of  the  tangential  residual  stresses  is 
taken  from  the  hole  wall  to  the  edge  of  the  part.  These  residual  stresses  are  input  into 
LifeWorks  as  a  residual  stress  gradient  and  used  in  the  analysis.  The  flaw  then  starts  from 
the  standard  0.05”  but  has  the  benefit  of  residual  stresses  included.  See  Figure  A-5  below. 
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StressCheck  9.2.0 
lits  =  INCH/LBF/SEC/ 
NLMAT  ID=CW_1_2_2 
Run=3  ,  DOF=3440 
Fnc. =Sy  (L-Cyl.12) 
Max=  2.188e+004 
Hi n=-5 . 870e+004 


2 . 188e+004 
1. 382e+004 
5 . 760e+003 
-2 . 298e+003 
-1. 03Se+004 
-1. 841e+004 
-2 . 647e+004 
-3 . 45  3  e  +  004 
-4. 2  5  9e  +  004 
-5 . 06Ee+004 
-5 . 870e+004 


Figure  A-5.  Stress  Check  Result 

A.3  RBDMS  Modifications 

1 .  SFPOF  system  reliability  calculation  formula  has  been  modified. 

a.  For  the  system  reliability  calculation,  the  RBDMS  code  applied  the  same  formula 
used  by  the  PROF  code.  The  simple  formula  used  was  to  multiply  the  calculated 
risk  with  the  number  of  locations;  this  is  actually  an  approximation  formula  and  it 
is  only  good  for  the  problems  with  smaller  risk.  When  the  risk  is  getting  larger, 
this  approximation  formula  may  be  incorrect  and  it  can  yield  unreasonable 
SFPOF  value  (i.e.,  >1). 

b.  To  modify  the  potential  mistake,  the  RBDMS  code  has  been  modified  by  using 
the  following  formula: 

SFPOF(system)  =  1  -  (1  -  SFPOF(component))A(number  of  Similar  Locations) 

2.  For  each  location  (control  point),  more  than  one  inspection  type  (i.e.,  multiple  PODs)  for 
both  field  inspection  and  program  depot  maintenance  (PDM)  inspection  can  be 
considered  and  input. 

a.  The  purpose  of  developing  this  capability  is  to  accommodate  potential  multiple 
inspection  types  for  the  same  location.  A  potential  scenario  is  to  use  a  POD  with 
lower  fidelity  to  detect  the  cracks  during  the  field  inspection  and  then  use  a  POD 
with  higher  fidelity  to  detect  the  cracks  during  PDM.  This  capability  will  greatly 
enhance  the  reality  of  our  cost  analysis  model  by  accurately  calculating  the 
Percentage  of  Cracks  Detected  (PCD)  data  for  various  PODs  used. 

b.  To  accommodate  this  capability,  the  RBDMS  code’s  “POD”  input  file  format  has 
been  modified  and  the  PODs  must  be  properly  stored  for  the  computation  of  the 
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PCD.  The  math  computation  method  and  code  for  the  PCD  remain  unchanged 
because  only  POD  parameters  have  been  modified. 

3.  For  each  inspection,  more  than  one  POD  types  can  be  selected  and  used  to  compute  the 
percentage  of  crack  detected  (PCD). 

a.  The  purpose  of  developing  this  capability  is  similar  to  those  described  in  item  2 
above. 

b.  To  accommodate  this  capability,  the  RBDMS  code’s  “inspection”  input  file 
format  has  been  modified  and  the  “inspection”  options  must  be  properly  stored  for 
computation  of  the  PCD.  The  math  computation  method  and  code  for  the  PCD 
remain  unchanged  because  only  POD  parameters  have  been  modified. 

4.  For  each  inspection,  the  PCD  will  be  calculated  and  subdivided  into  small,  median,  and 
large  crack  zones  so  different  repair  strategies  can  be  applied. 

a.  The  purpose  of  this  new  capability  is  to  compute  the  PCDs  for  the  various  repair 
strategies.  When  the  identified  crack  size  is  smaller,  a  simple  reaming  process 
can  be  applied;  however,  when  the  crack  size  is  larger,  a  more  costly  or  time- 
consuming  repair  strategy  needs  to  be  applied.  With  this  capability,  the  repair 
cost  can  be  calculated  with  higher  fidelity. 

b.  For  this  new  capability,  the  user  must  determine  two  crack  sizes  (al  and  a2)  and 
input  these  data  for  the  RBDMS  code  to  compute  the  PCDSmall  (a  <  al), 
PCDMedian  (al  <  a  <  al),  and  PCDLarge  (a  >  a2).  The  input  file  of  the  RBDMS 
code  needs  to  be  changed  accordingly  to  input  the  two  crack  sizes.  The  code  to 
calculate  the  PCD  has  been  modified  to  subdivide  the  original  calculated  PCD 
into  three  pieces  as  described  above  by  properly  managing  the  PCD  data.  The 
math  computation  method  and  code  for  the  PCD  remain  unchanged. 

5.  A  new  summary  output  file  with  all  the  necessary  data  for  the  cost  analysis  has  been 
defined  and  developed. 

a.  The  purpose  of  this  summary  file  is  to  provide  a  tabular  format  of  all  the  key 
output  data  so  it  can  be  easily  read  in  by  the  cost  analysis  code.  Previously,  the 
cost  analysis  code  had  to  read  in  several  output  files. 

b.  To  record  these  data,  only  some  bookkeeping  of  these  output  data  are  required 
and  there  are  no  changes  for  the  math  computation  method  and  code.  A  new 
RBDMStable.out  has  been  created  to  store  these  data  by  the  RBDMS  code. 

6.  The  crack  growth  and  geometry  data  input  files  have  been  modified  for  the  RBDMS 
code. 

a.  At  present,  to  run  the  RBDMS  code,  it  is  necessary  to  modify  both  the  crack 
growth  and  geometry  data  files  by  inserting  two  additional  lines  and  modifying 
one  line.  The  first  line  is  to  add  a  line  for  the  total  no.  of  data  pairs  and  the  second 
line  is  to  add  a  line  of  (0.,  0.)  as  the  starting  point  for  the  crack  growth 
computation.  Then,  it  needs  to  modify  the  original  first  line  of  the  input  data  (al, 
tl)  and  change  it  to  (al,  tl/2).  Therefore,  the  purpose  of  this  task  is  to  reduce  the 
modifications  as  much  as  possible  so  it  will  be  easier  for  the  user  to  use  the 
RBDMS  code. 
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b.  To  meet  the  needs,  the  RBDMS  code  has  been  modified  to  accept  both  the  crack 
growth  and  the  geometry  files  with  only  one  modification  still  needed.  For  both 
the  crack  growth  and  the  geometry  files,  the  user  only  needs  to  input  the  line  with 
the  total  no.  of  data  pairs.  Without  this  line,  the  RBDMS  code  will  have  difficulty 
reading  in  the  data  pairs  correctly.  There  are  no  changes  for  the  math 
computation  method  and  code  and  the  modified  code  has  the  capability  to  read  in 
the  following  two  options: 

i.  Option  one  will  be  the  current  case:  For  the  crack  growth  data  file,  the 
user  needs  to  add  a  line  of  the  total  number  of  input  data  pairs,  a  data  pair 
(0.,  0.),  and  then  modify  the  original  first  line  of  the  input  data  as 
described  above.  For  the  geometry  file,  user  also  needs  to  input  the  total 
number  of  input  data  pairs,  a  data  pair  (0.,  0.),  and  then  modify  the 
original  first  line  of  the  input  data  as  described  above. 

ii.  Option  two  will  be  the  newer  option.  For  this  option,  user  only  needs  to 
input  a  line  of  the  total  number  of  input  data  pairs  for  both  the  crack 
growth  data  file  and  the  geometry  file.  There  is  no  need  to  add/modify  the 
other  data. 

7.  A  computation  error  occurred  for  both  RBDMS  and  PROF  codes  when  the  EIFS 
distribution  has  the  crack  size  larger  than  the  critical  crack  length  used  in  the  crack 
growth  curve  data  (for  example,  DTAs  130,  131,  133A,  and  145).  The  RBDMS  code  has 
been  modified  to  resolve  this  problem  and  issue  a  warning  sign  for  this  type  of  problems. 

a.  When  this  type  of  problem  has  been  identified,  first,  a  warning  message  will  be 
shown  in  the  output  data  file  to  advise  user  to  check  their  input  data  especially  the 
EIFS  and  CG  curve  data.  Normally,  the  EIFS  data  should  be  much  smaller  than 
the  critical  crack  length. 

b.  For  the  RBDMS  code  to  resolve  this  problem,  an  artificial  data  pair  with  the  crack 
length  longer  than  the  critical  crack  length  was  added  for  the  risk  computation 
purpose.  A  new  limit  for  the  artificial  crack  length  has  been  modified  so  it  can  be 
automatically  increased  to  avoid  this  type  of  errors,  i.e.,  when  the  initial  crack  size 
is  larger  than  the  critical  crack  length.  The  potential  problem  for  this  modification 
is  that  the  extended  crack  length  can  be  unreasonable  compared  to  the  actual 
structure  data.  That  is  the  reason  why  it  needs  to  provide  the  warning  message  for 
user  to  check  the  input  data.  The  math  computation  method  and  code  for  the 
SFPOF  remain  unchanged  because  of  this  modification. 

8.  A  time  zero  SFPOF  calculation  has  been  implemented  so  it  can  be  used  in  cost  analysis. 

a.  The  purpose  is  to  compute  the  time  zero  SFPOF  so  user  can  compare  if  this  is 
starting  from  a  very  small  initial  crack  size  distribution.  It  is  a  good  starting  data 
point  for  the  cost  analysis  as  well. 

b.  To  accommodate  this  capability,  the  input  file  will  not  be  changed  but  the  code 
has  been  modified  so  it  can  automatically  compute  the  time  zero  SFPOF.  The 
math  computation  method  and  code  for  this  new  capability  remain  unchanged. 

The  key  is  to  properly  manage  the  code  by  introducing  an  additional  computation 
of  SFPOF  at  time  zero  where  the  initial  crack  size  distribution  will  be  used  as  the 


77 

Approved  for  public  release;  distribution  unlimited. 


crack  size  distribution  for  the  SFPOF  computation.  The  output  file  for  this  new 
capability  has  been  modified  accordingly. 


9.  The  output  file  names  have  been  modified  so  it  can  be  read  easily. 

a.  Change  the  other  output  file  names  so  it  can  be  read  easily.  The  RBDMS  code 
has  been  modified  accordingly.  The  math  computation  method  and  code  for  this 
new  capability  remain  unchanged. 

b.  The  following  output  file  names  have  been  changed: 

i.  Plotl.out  to  RBDMSdists.out 

ii.  Plot2.out  to  SFPOF. out 

iii.  Plot3.out  to  PCDall.out 

iv.  Plot3 1  .out  to  PCDsmall.out 

v.  Plot32.out  to  PCDlarge.out 

vi.  Plotl  1  .out  to  RBDMStable.out 
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